| Age | Commit message (Collapse) | Author |
|---|
|
Using the thunk->dispatch() method directly as the iou_op cb leaks
the thunk
|
|
s/quadratically/exponentially/
|
|
libiou got bulk ops allocating and free lists
|
|
Take advantage of the registered fds.
|
|
This introduces an journal_t.idx member for "fixed" style usage
in io_uring parlance. Once all journals get opened, they're registered
in the underlying iou ring.
Subsequent operations on these files may supply these idx values
as the fd, but must also add IOSQE_FIXED_FILE to the sqe's flags.
This commit also switches the already present read operations to
use the idx/IOSQE_FIXED_FILE method.
Theoretically this should offer some efficiency gains, since the
kernel can now skip some per-operation fd handling overheads by
having them done once instead @ registration time.
|
|
In order to make use of the liburing register_files() helper, the
struct ring pointer is needed.
|
|
The bucketized counts already have %ages, but not the overall
count.
|
|
This doesn't make use of all the parameters it inherited from
journal_iter_objects_dispatch(), so at least get rid of them if
you're not going to just bite the bullet and refactor this thing
out of the picture.
|
|
This gives some visibility into EntryArrayObject duplication and
utilization statistics.
It's not the tidiest of code, just something I slapped together
last night.
|
|
Preparatory commit for report-entry-arrays, I used libcrypto for
SHA1 "perfect" hashing of the payloads. This way I didn't need
to keep the payloads themselves around for counting duplicates,
the SHA1 digests suffice.
|
|
As-is this doesn't update the header's arena_size correctly, so
disable it to prevent unsuspecting users from producing journals
that journald thinks are corrupt.
|
|
fclose() implicitly calls fflush()...
|
|
Reserve upper case just for table type objects like arrays and
hash tables.
|
|
This is derived from report-usage.c, tidy this up.
|
|
This writes a .layout file for every opened journal, which
describes the sequential object layout for the respective
journal.
Sample output:
```
Layout for "user-1000.journal"
Legend:
? OBJECT_UNUSED
d OBJECT_DATA
f OBJECT_FIELD
e OBJECT_ENTRY
D OBJECT_DATA_HASH_TABLE
F OBJECT_FIELD_HASH_TABLE
A OBJECT_ENTRY_ARRAY
T OBJECT_TAG
|N| object spans N page boundaries (page size used=4096)
| single page boundary
+N N bytes of alignment padding
+ single byte alignment padding
F|5344 D|448|1834896 d81+7 f50+6 d74+6 f48 d82+6 f55+ d84+4 f57+7 d80 f50+6 d122+6 f47+ d74+6 f44+4 d73+7 f44+4 d70+2 f44+4 d72 f45+3 d76+4 f44+4 d75+5 f48 d90+6 f54+2 d80 f54+2 d84+4 f55+ d123+5 f55+ d82+6 f56 d87+ f58+6 d93+3 f53+3 d|94+2 f54+2 d91+5 f59+5 d119+ f62+2 d107+5 f66+6 d105+7 f48 d108+4 f51+5 d82+6 f49+7 e480 A56 d80 d104 d74+6 d73+7 d107+5 e480 A56 A56 A56 A56 A56 A56 A56 A56 A56 A56 A56 A56 A56 A56 A56 A56 A56 A56 A56 A56 A56 d97+7 d107+5 e|480 A56 A56 A56 d136 d107+5 e480 A56 d74+6 d148+4 d107+5 e480 A88 d107+5 e480 A88 A88 A88 A56 A88 A88 A88 A88 A88 A88 A88 A88 A88 A88 A|88 A88 A88 A88 A88 A88 A88 d80 d74+6 d107+5 e480 A88 A56 d107+5 e480 A56 A56 A56 d107+5 e480 A56 d107+5 e480 A88 A56 A56 d107+5 e|480 d80 d74+6 d107+5 e480 d97+7 d107+5 e480 A232 A88 A56 A56 d142+2 d107+5 e480 A232 A232 A232 A232 A232 A|232 A232 A232 A232 A232 A232 A232 A232 A232 A232 A232 A232 A232 A232 A232 d107+5 e480 d107+5 e|480 d80 d74+6 d107+5 e480 A232 d107+5 e480 A56 A56 d107+5 e480 d107+5 e480 d107+5 e304 d80 d74+6 d107+5 e|480 d107+5 e480 A56 A56 d107+5 e480 A232 d107+5 e480 d107+5 e480 A88 d80 d74+6 d107+5 e480 A88 d107+5 e|480 A56 A56 d107+5 e480 d107+5 e480 A88 A88
```
This provides insight into the distribution of object types, how
much space is spent on alignment padding, how frequently objects
land on page boundaries - something preferably avoided especially
for small objects near a boundary.
In the above example, we can immediately make the observation
that early in the journal there are interleaved data and field
objects, and this cluster of the initially added data and field
objects gets torn by a page boundary. It's possible if they
weren't interleaved when adding an entry and its respective data
+ fields, fitting all the early (common) data objects within a
single page might confer some performance gain. The field
objects aren't currently used by data object heavy operations, it
doesn't make much sense to have them polluting the area of the
initially added data objects or really any otherwise clustered
data objects.
|
|
Make naming a bit more descriptive and consistent...
journals_for_each() is a simple non-IO-incurring journals
array iteration, journal_for_each() generates IO and walks
the data objects hash table... they're quite different,
and shouldn't have such similar names.
|
|
|
|
Now there's even instructions and legalese
|
|
Now it's easy to build even
|
|
This is a very quick and dirty experimental hack written in some
sort of bastard continuation-passing style in C w/io_uring using
journal-file introspection and manipulation duty as an excuse for
its existence.
Consider this unfinished prototype quality code.
|
|
This brings journal-file data structures and byte swapping
helpers I may as well just reuse.
I've had to do some minor messing about to make things workable in
isolation out of the systemd tree without pulling in too much.
I've also added a new HashedObjectHeader type to encompass the
slightly larger common header components of FieldObject and
DataObject to facilitate a generic hash table iterator that can
operate on just loading HashedObjectHeader when nothing more
than the object size is needed for accounting. Basically the
HashedObjectHeader is ObjectHeader+hash+next_hash_offset.
|
|
libiou is a thin veneer over io_uring defining an ergonomic async
IO-oriented API.
Since jio shouldn't need to do anything computationally
intensive, the combination of thunk_h for closures and
libiou+io_uring for scheduling closures according to IO
completions should be sufficient for an implementation. Though
it may prove annoying to not have per-task stacks and the ability
to arbitrarily yield and/or delay execution pending completion of
non-IO results, we'll see.
|
|
thunk_h makes using closure-esque callbacks in C somewhat
convenient.
|
