Skip to content
This repository was archived by the owner on Feb 18, 2025. It is now read-only.

Commit 113809a

Browse files
chasesynoet
andauthored
Fix some expanded storage bugs (#82)
* Ensure vector contents stay alive when in use, fix read/write flags * Return part content as uint8, doesn't actually impact functionality, but makes debugging easier * Add better relationship handling for docx export (#84) * Add debug/dev build, update instructions * Split compile and link flags for dev/debug mode --------- Co-authored-by: teo <[email protected]>
1 parent 85ea8c6 commit 113809a

File tree

21 files changed

+756
-237
lines changed

21 files changed

+756
-237
lines changed

.github/workflows/smoketest.yml

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -141,7 +141,7 @@ jobs:
141141
export CCACHE_COMPRESSLEVEL=6
142142
export CCACHE_MAXSIZE=5G
143143
(cd libreoffice-core/desktop/wasm && npm install --loglevel=error)
144-
./scripts/configure && ./scripts/build
144+
./scripts/configure dev && ./scripts/build
145145
146146
- name: Cache external tarballs
147147
uses: runs-on/cache/save@v4

README.md

Lines changed: 59 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -8,9 +8,35 @@ This project is not a part of the official LibreOffice project, nor endorsed by
88

99
# Prerequisites
1010

11-
TODO: List pre-reqs for each platform
11+
You must use Linux on x64, otherwise the build will not work.
12+
13+
On Ubuntu/Debian/Pop_OS!, you should install these packages:
14+
15+
```bash
16+
apt-get install -y --no-install-recommends \
17+
git \
18+
build-essential \
19+
zip \
20+
nasm \
21+
python3 \
22+
python3-dev \
23+
autoconf \
24+
gperf \
25+
xsltproc \
26+
libxml2-utils \
27+
bison \
28+
flex \
29+
pkg-config \
30+
ccache \
31+
openssh-server \
32+
cmake \
33+
sudo \
34+
locales \
35+
libnss3
36+
```
1237

1338
Setup the repo:
39+
1440
```bash
1541
git clone https://github.com/coparse-inc/lok-wasm
1642
./scripts/setup
@@ -19,14 +45,42 @@ git clone https://github.com/coparse-inc/lok-wasm
1945
# Building
2046

2147
```bash
22-
# Run configure.sh for the initial build or any configuration changes
23-
./scripts/configure
48+
# Run configure for the initial build or any configuration changes
49+
./scripts/configure dev
2450
# if you're using VS Code or clangd in vim, run this
2551
./scripts/clangd
26-
# Run build.sh for any code changes
52+
# Run build for any code changes
2753
./scripts/build
2854
```
2955

56+
# Debugging
57+
58+
Make a debug build:
59+
60+
```bash
61+
# Clean the existing build
62+
(cd libreoffice-core/ && make clean)
63+
# Run configure for debug
64+
./scripts/configure debug
65+
# Run build for any code changes
66+
./scripts/build
67+
```
68+
69+
Use Chrome with [the C/C++ WASM debugging tools extension](https://goo.gle/wasm-debugging-extension) installed.
70+
71+
Use Ctrl/Cmd+P in the Dev Tools to quickly navigate to the `.cxx` file you need to debug.
72+
73+
You can add breakpoints as necessary in the Dev Tools, but conditional breakpoints still aren't supported.
74+
If you need to add a conditional breakpoint, use an if statement with a log to set a break point on inside of your code:
75+
```
76+
if (myCondition == true) {
77+
SAL_WARN("debug", "here");
78+
}
79+
```
80+
81+
82+
Expand the LOK logs to see the full stack trace, clicking on the `.cxx` file will jump to the source for the file.
83+
3084
# QA Env
3185

3286
Make a build first, then:
@@ -43,5 +97,6 @@ npm run dev
4397
# Docs
4498

4599
<!-- TIP: in neovim, you can use `gf` to go to the file linked if the cursor is between ( ) -->
100+
46101
- [Remote Development](./remote_dev.md)
47102
- [Important Files](./important_files.md)

VERSION

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1 +1 @@
1-
v0.1.0
1+
v0.1.1

important_files.md

Lines changed: 7 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -25,4 +25,11 @@ The LibreOffice "core" from which LOK is built, forked from upstream with propri
2525
[`desktop/inc/lib/init.hxx`](./libreoffice-core/desktop/inc/lib/init.hxx)
2626
[`desktop/source/lib/init.cxx`](./libreoffice-core/desktop/source/lib/init.cxx) - the implementation for the original LOK bindings, don't add new things here or in `LibreOfficeKit.hxx/h`
2727

28+
29+
#### Expanded Storage
30+
[`libreoffice-core/oox/source/helper/expandedstorage.cxx`](./libreoffice-core/oox/source/helper/expandedstorage.cxx) - ExpandedStorage implementation that allows for loading/saving docx files in memory using their expanded parts
31+
[`libreoffice-core/comphelper/source/misc/relationshipaccess.cxx`](./libreoffice-core/comphelper/source/misc/relationshipaccess.cxx) - relationship access for file streams in expanded storage
32+
[`libreoffice-core/comphelper/source/streaming/vecstream.cxx`](./libreoffice-core/comphelper/source/streaming/vecstream.cxx) - implementation for vector input/output streams used in expanded storage
33+
34+
2835
## `qa-env`

libreoffice-core/comphelper/source/misc/relationshipaccess.cxx

Lines changed: 11 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,4 +1,8 @@
1+
#include <comphelper/ofopxmlhelper.hxx>
2+
#include <vector>
3+
#include <comphelper/processfactory.hxx>
14
#include <comphelper/relationshipaccess.hxx>
5+
#include <comphelper/vecstream.hxx>
26
#include <com/sun/star/container/NoSuchElementException.hpp>
37
#include <com/sun/star/embed/InvalidStorageException.hpp>
48
#include <com/sun/star/embed/StorageWrappedTargetException.hpp>
@@ -25,6 +29,12 @@ const beans::StringPair* lcl_findPairByName(const Sequence<beans::StringPair>& r
2529
[&rName](const beans::StringPair& rPair) { return rPair.First == rName; });
2630
}
2731

32+
void RelationshipAccessImpl::setRelationships(
33+
css::uno::Sequence<css::uno::Sequence<css::beans::StringPair>> aRelInfo)
34+
{
35+
m_aRelInfo = std::move(aRelInfo);
36+
}
37+
2838
void SAL_CALL RelationshipAccessImpl::clearRelationships() { m_aRelInfo.realloc(0); }
2939

3040
void SAL_CALL RelationshipAccessImpl::insertRelationships(
@@ -84,7 +94,7 @@ void SAL_CALL RelationshipAccessImpl::removeRelationshipByID(const OUString& sID
8494
auto nInd = static_cast<sal_Int32>(std::distance(std::cbegin(aSeq), pRel));
8595
comphelper::removeElementAt(aSeq, nInd);
8696

87-
m_aRelInfo = aSeq;
97+
m_aRelInfo = std::move(aSeq);
8898

8999
// TODO/LATER: in future the unification of the ID could be checked
90100
return;

libreoffice-core/comphelper/source/streaming/vecstream.cxx

Lines changed: 96 additions & 22 deletions
Original file line numberDiff line numberDiff line change
@@ -1,3 +1,5 @@
1+
#include <com/sun/star/io/XOutputStream.hpp>
2+
#include <sal/log.hxx>
13
#include <com/sun/star/embed/XRelationshipAccess.hpp>
24
#include <com/sun/star/io/BufferSizeExceededException.hpp>
35
#include <com/sun/star/io/NotConnectedException.hpp>
@@ -15,13 +17,14 @@ using namespace ::com::sun::star::io;
1517
using namespace ::com::sun::star::uno;
1618
using namespace ::osl;
1719

18-
VectorInputStream::VectorInputStream(std::vector<sal_Int8>& vec)
20+
// ----------- VectorInputStream -----------
21+
VectorInputStream::VectorInputStream(std::shared_ptr<std::vector<sal_Int8>> vec)
1922
: m_vec(vec)
2023
, m_pos(0)
2124
{
2225
}
2326

24-
sal_Int32 SAL_CALL VectorInputStream::available() { return m_vec.size() - m_pos; }
27+
sal_Int32 SAL_CALL VectorInputStream::available() { return m_vec->size() - m_pos; }
2528

2629
void SAL_CALL VectorInputStream::closeInput() {}
2730

@@ -35,13 +38,13 @@ sal_Int32 SAL_CALL VectorInputStream::readBytes(css::uno::Sequence<sal_Int8>& da
3538

3639
std::scoped_lock gaurd(m_mutex);
3740

38-
sal_Int32 avail = m_vec.size() - m_pos;
41+
sal_Int32 avail = m_vec->size() - m_pos;
3942

4043
if (avail < count)
4144
count = avail;
4245

4346
data.realloc(count);
44-
memcpy(data.getArray(), m_vec.data() + m_pos, count);
47+
memcpy(data.getArray(), m_vec->data() + m_pos, count);
4548
m_pos += count;
4649

4750
return count;
@@ -52,12 +55,12 @@ sal_Int32 VectorInputStream::readSomeBytes(sal_Int8* data, sal_Int32 count)
5255
throw BufferSizeExceededException(OUString(), *this);
5356

5457
std::scoped_lock gaurd(m_mutex);
55-
sal_Int32 avail = m_vec.size() - m_pos;
58+
sal_Int32 avail = m_vec->size() - m_pos;
5659

5760
if (avail < count)
5861
count = avail;
5962

60-
memcpy(data, m_vec.data() + m_pos, count);
63+
memcpy(data, m_vec->data() + m_pos, count);
6164
m_pos += count;
6265

6366
return count;
@@ -75,7 +78,7 @@ void SAL_CALL VectorInputStream::skipBytes(sal_Int32 skip)
7578

7679
std::scoped_lock aGuard(m_mutex);
7780

78-
sal_Int32 avail = m_vec.size() - m_pos;
81+
sal_Int32 avail = m_vec->size() - m_pos;
7982

8083
if (avail < skip)
8184
skip = avail;
@@ -85,7 +88,7 @@ void SAL_CALL VectorInputStream::skipBytes(sal_Int32 skip)
8588

8689
void SAL_CALL VectorInputStream::seek(sal_Int64 location)
8790
{
88-
if (location > (sal_Int64)m_vec.size() || location < 0 || location > SAL_MAX_INT32)
91+
if (location > (sal_Int64)m_vec->size() || location < 0 || location > SAL_MAX_INT32)
8992
throw IllegalArgumentException("bad location", static_cast<cppu::OWeakObject*>(this), 1);
9093
std::scoped_lock gaurd(m_mutex);
9194
m_pos = static_cast<sal_Int32>(location);
@@ -100,10 +103,40 @@ sal_Int64 SAL_CALL VectorInputStream::getPosition()
100103
sal_Int64 SAL_CALL VectorInputStream::getLength()
101104
{
102105
std::scoped_lock gaurd(m_mutex);
103-
return m_vec.size();
106+
return m_vec->size();
107+
}
108+
Any SAL_CALL VectorInputStream::queryInterface(const Type& rType)
109+
{
110+
Any aRet = cppu::queryInterface(rType, static_cast<embed::XRelationshipAccess*>(this),
111+
static_cast<lang::XTypeProvider*>(this),
112+
static_cast<io::XInputStream*>(this));
113+
if (aRet.hasValue())
114+
return aRet;
115+
116+
return OWeakObject::queryInterface(rType);
117+
}
118+
Sequence<Type> SAL_CALL VectorInputStream::getTypes()
119+
{
120+
static css::uno::Sequence<css::uno::Type> aTypes = {
121+
cppu::UnoType<css::lang::XTypeProvider>::get(),
122+
cppu::UnoType<embed::XRelationshipAccess>::get(),
123+
cppu::UnoType<io::XInputStream>::get(),
124+
};
125+
return aTypes;
104126
}
105127

106-
VectorOutputStream::VectorOutputStream(std::vector<sal_Int8>& vec)
128+
Sequence<sal_Int8> SAL_CALL VectorInputStream::getImplementationId()
129+
{
130+
return Sequence<sal_Int8>();
131+
}
132+
133+
void SAL_CALL VectorInputStream::acquire() noexcept { OWeakObject::acquire(); }
134+
135+
void SAL_CALL VectorInputStream::release() noexcept { OWeakObject::release(); }
136+
137+
// ----------- VectorOutputStream -----------
138+
139+
VectorOutputStream::VectorOutputStream(std::shared_ptr<std::vector<sal_Int8>> vec)
107140
: m_vec(vec)
108141
, m_pos(0)
109142
{
@@ -112,21 +145,66 @@ VectorOutputStream::VectorOutputStream(std::vector<sal_Int8>& vec)
112145
void SAL_CALL VectorOutputStream::writeBytes(const Sequence<sal_Int8>& data)
113146
{
114147
std::scoped_lock gaurd(m_mutex);
115-
sal_Int32 available = m_vec.size() - m_pos;
148+
sal_Int32 available = m_vec->size() - m_pos;
116149
if (available < data.getLength())
117150
{
118151
std::size_t newSize = static_cast<std::size_t>(m_pos + data.getLength());
119-
m_vec.resize(newSize);
152+
m_vec->resize(newSize);
120153
}
121-
memcpy(m_vec.data() + m_pos, data.getConstArray(), data.getLength());
154+
memcpy(m_vec->data() + m_pos, data.getConstArray(), data.getLength());
122155
m_pos += data.getLength();
123156
}
124157

125-
void SAL_CALL VectorOutputStream::flush() {}
126-
void SAL_CALL VectorOutputStream::closeOutput() {}
158+
void SAL_CALL VectorOutputStream::flush()
159+
{
160+
// if the vector is the right size, this is a no-op, if it's writing over an existing stream, it gets truncated to end of the last write
161+
m_vec->resize(m_pos);
162+
}
163+
164+
void SAL_CALL VectorOutputStream::closeOutput()
165+
{
166+
// see ::flush() for why
167+
m_vec->resize(m_pos);
168+
}
169+
170+
Any SAL_CALL VectorOutputStream::queryInterface(const Type& rType)
171+
{
172+
Any aRet = cppu::queryInterface(rType, static_cast<embed::XRelationshipAccess*>(this),
173+
static_cast<lang::XTypeProvider*>(this),
174+
static_cast<io::XOutputStream*>(this));
175+
if (aRet.hasValue())
176+
return aRet;
177+
178+
return OWeakObject::queryInterface(rType);
179+
}
180+
Sequence<Type> SAL_CALL VectorOutputStream::getTypes()
181+
{
182+
static css::uno::Sequence<css::uno::Type> aTypes = {
183+
cppu::UnoType<css::lang::XTypeProvider>::get(),
184+
cppu::UnoType<embed::XRelationshipAccess>::get(),
185+
cppu::UnoType<io::XOutputStream>::get(),
186+
};
187+
return aTypes;
188+
}
189+
190+
Sequence<sal_Int8> SAL_CALL VectorOutputStream::getImplementationId()
191+
{
192+
return Sequence<sal_Int8>();
193+
}
194+
195+
void SAL_CALL VectorOutputStream::acquire() noexcept { OWeakObject::acquire(); }
196+
197+
void SAL_CALL VectorOutputStream::release() noexcept { OWeakObject::release(); }
127198

128-
VecStreamSupplier::VecStreamSupplier(Reference<io::XInputStream> inputStream,
129-
Reference<io::XOutputStream> outputStream)
199+
VecStreamContainer::VecStreamContainer(Reference<VecStreamSupplier>& stream)
200+
: m_stream(stream)
201+
{
202+
}
203+
204+
// ----------- VecStreamSupplier -----------
205+
206+
VecStreamSupplier::VecStreamSupplier(Reference<VectorInputStream> inputStream,
207+
Reference<VectorOutputStream> outputStream)
130208
: m_inputStream(std::move(inputStream))
131209
, m_outputStream(std::move(outputStream))
132210
{
@@ -195,11 +273,7 @@ void SAL_CALL VecStreamSupplier::acquire() noexcept { OWeakObject::acquire(); }
195273

196274
void SAL_CALL VecStreamSupplier::release() noexcept { OWeakObject::release(); }
197275

198-
VecStreamContainer::VecStreamContainer(Reference<io::XStream>& stream)
199-
: m_stream(stream)
200-
{
201-
}
202-
276+
// ----------- VecStreamContainer -----------
203277
Reference<io::XInputStream> SAL_CALL VecStreamContainer::getInputStream()
204278
{
205279
return m_stream->getInputStream();

libreoffice-core/desktop/Executable_soffice_bin.mk

Lines changed: 0 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -91,7 +91,6 @@ $(eval $(call gb_Library_use_custom_headers,soffice_bin,\
9191
# See: https://github.com/emscripten-core/emscripten/blob/main/src/settings.js
9292
$(eval $(call gb_Executable_add_ldflags,soffice_bin,\
9393
--pre-js $(SRCDIR)/desktop/wasm/pre-js.js \
94-
--profiling \
9594
-lworkerfs.js -s MODULARIZE=1 -s EXPORT_NAME=LOK -s EXPORT_ES6=1 -s ENVIRONMENT=worker --no-entry \
9695
))
9796

0 commit comments

Comments
 (0)