| Age | Commit message (Collapse) | Author |
|---|
|
This restores the original til_fragmenter_slice_per_cpu() while
adding an explicit x16 variant for what
til_fragmenter_slice_per_cpu() had become.
The impetus for this is realizing the x16 multiplier is terrible
for sparkler's crappy threading, and it really need strictly
n_cpus slices for the threading to be beneficial.
So I'm just getting rid of the hidden x16 in favor of making it
explicit in an _x16 variant.
Subsequent commit will pivot all the non-sparkler callers to
til_fragmenter_slice_per_cpu_x16().
|
|
With the introduction of discoverable on-stream contexts, with
the intention of modules finding them at runtime for nested
rendering, relying exclusively on limiting n_cpus @
til_module_create_context() is no longer adequate.
When the nested rendering makes use of a context created
elsewhere, it can't make any assumptions about what its n_cpus
is, nor should it be attempting to change that value for its use.
So this variant just adds a max_cpus parameter for setting an
upper bound to whatever is found in the context's n_cpus.
The impetus for this is to fix the ref builtin, which you can
currently trick into deadlock by performing a nested threaded
render within a threaded render. What the ref module should be
doing is propagating its own context's n_cpus down as the upper
bound, via this new render variant.
|
|
This moves the core til_module_setup() functionality out into a
more composable helper with an eye towards all meta-modules using
it for doing their module setup instead of all the ad-hoc stuff
occurring presently.
Since til_module_setup_full() needs more paramaters to influence
its behavior, it's not directly usable as a "setup_func". But is
trivially wrapped into one, which is what til_module_setup() has
now become.
It's in the wrapper that users are expected to do specify what's
necessary for influencing til_module_setup_full() to do what's
appropriate for their needs.
|
|
Thin wrapper around gettimeofday(), prolly change the main ticks
stuff over to this too.
|
|
checkers::fill_module is presently implemented via creating
n_cpus identical module contexts, each having n_cpus=1.
Establishing a set of cloned per-cpu contexts, allowing threaded
rendering using any fill_module, regardless of if that module
internally implements threaded rendering.
This works fine, but creates some awkwardness for a future where
contexts are registered and discoverable within the stream at
their respective setup's path. In the checkers::fill_module
case, there would be n_cpus contexts at the same path since they
share the same til_setup_t. Those need to be dealt with
gracefully, ideally making the clones available to another
potentially clone-needing module (imagine a checkers::fill_module
transitionining to another checkers::fill_module situation, where
the transitioned-to fill_module refers to the context by path, it
should get all the contexts out of the stream as-is)
In this commit I'm adding a more formalized method of creating
multiple contexts from the same set of parameters, in preparation
for a future where module contexts get registered on the stream
at their til_setup_t.path. By putting the cloned creates behind
the til API it should at least be relatively trivial to get the
on-stream context registration to capture the multiple contexts.
|
|
Rather than creating an orphaned settings instance private to
til_module_setup_randomize(), the function now requires the
settings instance be provided.
The one remaining caller of this function is modules::rtv. Now
that rtv is responsible for creating the settings beforehand, and
the settings may be created with a path prefix, rtv gets its
til_module_context_t->setup.path prefixed for all the channel
settings.
Another improvement is now the channel settings instance gets
created from the module name as the settings string. So while
it's not yet possible to sparsely specify settings with others
being randomized, at least now when log_channels=on is in effect,
the printed args include the top-level channel module.
Having proper complete paths for the rtv channel modules is
especially visible in --print-paths output FYI.
An interesting test for exercising all this is:
```
$ src/rototiller --module=rtv,duration=0,context_duration=0,snow_module=none,channels=all,log_channels=on --print-pipes --defaults --go 2>/tmp/channels
in another terminal:
$ tail -F /tmp/channels
```
watch the chaos unfold
|
|
There's often a need to exclude specific modules, though it's
often a hacky kludge. It's something relied upon currently for
preventing dangerous recursion scenarios, which will likely get
fixed up more robustly in the future.
|
|
Composite modules that want to provide "all" aliases for modules
like rtv have to construct a comma-separated string of all module
names, usually filtered by some flags.
This helper does just that, but it does add yet another
open_memstream() user to revisit when those all get fixed up.
|
|
Mechanical rename just to make this consistent with
til_module_setup()/til_module_setup_finalize()
I should probably do a cleanup pass throughout the til APIs to
standardize on a subject-verb-object or subject-object-verb
order... Things have become a little inconsistent organically
over time
|
|
This changes til_setup_t* from optional to required for
til_module_context_t creation, while dropping the separate path
parameter construction and passing throughout.
|
|
Preparatory commit for til_module_create_context() requiring
setups even when there's no til_module_t.setup() method.
This helper will produce the minimal til_setup_t in such cases,
or hand off the task to til_module_t.setup() when present.
Note the need for passing res_setting and res_desc to
til_module_t.setup() despite not being a settings-construction
scenario. This is because of how modules using nested settings
tend to use res_setting for storing the current setting in
accessing the nested instance, which must still occur even when
just baking the complete setup.
It's expected that any composite/meta modules utilizing other
modules will use this helper to produce the baked setups, instead
of the ad-hoc direct calling of til_module_t.setup() they do
presently.
|
|
setup_func isn't formally defined for libtil, but
setup_interactively() defacto establishes it and
til_module_t.setup() reflects the same signature and calling
convention except with til_settings_t constified.
This change makes them all consistent in this regard, but there
should probably be a formal typedef added for the function.
The reason for constifying this is I don't want setup functions
directly manipulating the settings instance. In the case of
rototiller::setup_interactively() we ensure the stdio-based
interactive setup is always the side doing the manipulation of
the settings. For a libtil-user like glimmer, it's slightly
different beast with GTK+ in the loop, but by preventing the
setup_funcs from messing directly with the settings (instead
having to describe what they want done iteratively), the
front-end always gets its opportunity to maintain its state while
doing the described things.
Of course, this is mostly a lie, and within libtil the constified
til_settings_t gets cast away to modify it in places. But
keeping that limited to within libtil is tolerable IMO. We just
don't want to see such casts in module code.
|
|
With taps more or less fully implemented, it seems appropriate to
get rid of the stubbed out knobs for now.
Taps don't express the same things about range and usage knobs
aspired towards, but they don't preclude adding such things
either. But it seems clear that the way knobs were stubbed won't
be complementing taps as things stand currently to add those
aspects.
|
|
Let's make it so til_module_context_t as returned from
til_module_context_new() can immediately be freed via
til_module_context_free().
Previously it was only after the context propagated out to
til_module_context_create() that it could be freed that way, as
that was where the module member was being assigned.
With this change, and wiring up the module pointer into
til_module_t.create_context() as well for convenient providing to
til_module_context_new(), til_module_t.create_context() error
paths can easily cleanup via `return til_module_context_free()`
But this does require the til_module_t.destroy_context() be able
to safely handle partially constructed contexts, since the
mid-create failure freeing won't necessarily have all the members
initialized. There will probably be some NULL derefs to fix up,
but at least the contexts are zero-initialized @ new.
|
|
TIL_MODULE_HERMETIC:
There's likely to be some new modules that are more
orchestration style components having external
runtime dependencies. Think stuff like a sequencer
talking to GNU Rocket, or something that plays back
pattern data from external files.
Those would need a GNU Rocket process to talk to
somewhere, or input pattern file paths. So they
shouldn't participate in stuff like random rtv shows
unless they have some fallbacks for when the dependencies
are unavailable. For pattern data it's realistic to
include some builtin patterns to fallback on, but we're
not there yet. So this flag when specified should opt
out of things like rtv or checkers fill_module random
selections.
TIL_MODULE_EXPERIMENTAL:
Theres no current way to have knowingly
unstable/unfinished modules available in-tree for
development/collaboration purposes without having them
also make stuff like rtv unstable. Modules having this
flag set should be excluded from random inclusion without
a --experimental or some such runtime flag specified.
This commit only assigns values and names for the flags, it
doesn't implement anything.
|
|
This was mostly done out of convenience at the expense of turning
the fragment struct into more of a junk drawer.
But properly cleaning up owned stream pipes on context destroy
makes the inappropriateness of being part of til_fb_fragment_t
glaringly apparent.
Now the stream is just a separate thing passed to context create,
with a reference kept in the context for use throughout. Cleanup
of the owned pipes on the stream supplied to context create is
automagic when the context gets destroyed.
Note that despite there being a stream in the module context, the
stream to use is still supplied to all the rendering family
functions (prepare/render/finish) and it's the passed-in stream
which should be used by these functions. This is done to support
the possibility of switching out the stream frame-to-frame, which
may be interesting. Imagine doing things like a latent stream
and a future stream and switching between them on the fly for
instance. If there's a sequencing composite module, it could
flip between multiple sets of tracks or jump around multiple
streams with the visuals immediately flipping accordingly.
This should fix the --print-pipes crashing issues caused by lack
of cleanup when contexts were removed (like rtv does so often).
|
|
There needs to be a way to address module context instances
by name externally, in a manner complementary to settings and
taps.
This commit adds a string-based path to til_module_context_t, and
modifies til_module_create_context() to accept a parent path
which is then concatenated with the name of the module to produce
the module instance's new path.
The name separator used in the paths is '/' just like filesystem
paths, but these paths have no relationship to filesystems or
files.
The root module context creation in rototiller's main simply
passes "" as the parent path, resulting in a "/" root as one
would expect.
There are some obvious complications introduced here however:
- checkers in particular creates a context per cpu, simply using
the same seed and setup to try make the contexts identical at
the same ticks value. With this commit I'm simply passing the
incoming path as the parent for creating those contexts, but
it's unclear to me if that will work OK. With an eye towards
taps deriving their parent path from the context path, I guess
these taps would all get the same parent and hash to the same
value despite being duplicated. Maybe it Just Works, but one
thing is clear - there won't be any way to address the per-cpu
taps as-is. Maybe that's desirable though, there's probably
not much use in trying to control the taps at the CPU
granularity.
- when the recursive settings stuff lands, it should bring along
the ability to explicitly name settings blocks. Those names
should override the module name in constructing the path.
I've noted as such in the code.
- these paths probably need to be hashed @ initialization time
so there needs to be a hash function added to til, and a hash
value accompanying the name in the module context. It'd be
dumb to keep recomputing the hash when these paths get used
for hash table lookups multiple times per frame...
there's probably more I'm forgetting right now, but this seems
like a good first step.
fixup root path
|
|
Preparatory commit for enabling cloneable/swappable fragments
There's an outstanding issue with the til_fb_page_t submission,
see comments. Doesn't matter for now since cloning doesn't happen
yet, but will need to be addressed before they do.
|
|
til_setting_desc_t.random() and til_module_randomize_setup() now
take seeds.
Note they are not taking a pointer to a shared seed, but instead
receive the seed by value.
If a caller wishes the seed to evolve on every invocation into
these functions, it should simply insert a rand_r(&seed) in
producing the supplied seed value.
Within a given randomizer, the seed evolves when appropriate.
But isolating the effects by default seems appropriate, so
callers can easily have determinism within their respective scope
regardless of how much nested random use occurs.
|
|
modules/checkers w/fill_module=$module requires a consistent
mapping of cpu to fragnum since it creates a per-cpu
til_module_context_t for the fill_module.
The existing implementation for threaded rendering maximizes
performance by letting *any* scheduled to run thread advance
fragnum atomically and render the acquired fragnum
indiscriminately. A side effect of this is any given frame, even
rendered by the same module, will have a random mapping of
cpus/threads to fragnums.
With this change, the simple til_module_t.prepare_frame() API of
returning a bare fragmenter function is changed to instead return
a "frame plan" in til_frame_plan_t. Right now til_frame_plan_t
just contains the same fragmenter as before, but also has a
.cpu_affinity member for setting if the frame requires a stable
relationship of cpu/thread to fragnum.
Setting .cpu_affinity should be avoided if unnecessary, and that
is the default if you don't mention .cpu_affinity at all when
initializing the plan in the ergonomic manner w/designated
initializers. This is because the way .cpu_affinity is
implemented will leave threads spinning while they poll for
*their* next fragnum using atomic intrinsics. There's probably
some room for improvement here, but this is good enough for now
to get things working and correct.
|
|
- modules now allocate their contexts using
til_module_context_new() instead of [cm]alloc().
- modules simply embed til_module_context_t at the start of their
respective private context structs, if they do anything with
contexts
- modules that do nothing with contexts (lack a create_context()
method), will now *always* get a til_module_context_t supplied
to their other methods regardless of their create_context()
presence. So even if you don't have a create_context(), your
prepare_frame() and/or render_fragment() methods can still
access seed and n_cpus from within the til_module_context_t
passed in as context, *always*.
- modules that *do* have a create_context() method, implying they
have their own private context type, will have to cast the
til_module_context_t supplied to the other methods to their
private context type. By embedding the til_module_context_t at
the *start* of their private context struct, a simple cast is
all that's needed. If it's placed somewhere else, more
annoying container_of() style macros are needed - this is
strongly discouraged, just put it at the start of struct.
- til_module_create_context() now takes n_cpus, which may be set
to 0 for automatically assigning the number of threads in its
place. Any non-zero value is treated as an explicit n_cpus,
primarily intended for setting it to 1 for single-threaded
contexts necessary when embedded within an already-threaded
composite module.
- modules like montage which open-coded a single-threaded render
are now using the same til_module_render_fragment() as
everything else, since til_module_create_context() is accepting
n_cpus.
- til_module_create_context() now produces a real type, not void
*, that is til_module_context_t *. All the other module
context functions now operate on this type, and since
til_module_context_t.module tracks the module this context
relates to, those functions no longer require both the module
and context be passed in. This is especially helpful for
compositing modules which do a lot of module context creation
and destruction; the module handle is now only needed to create
the contexts. Everything else operating on that context only
needs the single context pointer, not module+context pairs,
which was unnecessarily annoying.
- if your module's context can be destroyed with a simple free(),
without any deeper knowledge or freeing of nested pointers, you
can now simply omit destroy_context() altogether. When
destroy_context() is missing, til_module_context_free() will
automatically use libc's free() on the pointer returned from
your create_context() (or on the pointer that was automatically
created if you omitted create_context() too, for the
bare til_module_context_t that got created on your behalf
anyways).
For the most part, these changes don't affect module creation.
In some ways this eases module creation by making it more
convenient access seed and n_cpus if you had no further
requirement for a context struct.
In other ways it's slightly annoying to have to do type-casts
when you're working with your own context type, since before it
was all void* and didn't require casts when assigning to your
typed context variables.
The elimination for requiring a destroy_context() method in
simple free() of private context scenarios removes some
boilerplate in simple cases.
I think it's a wash for module writers, or maybe a slight win for
the simple cases.
|
|
In the recent surge of ADD-style rtv+compose focused development,
a bunch of modules were changed to randomize initial states at
context_create() so they wouldn't be so repetitive.
But the way this was done in a way that made it impossible to
suppress the randomized initial state, which sometimes may be
desirable in compositions. Imagine for instance something like
the checkers module, rendering one module in the odd cells, and
another module into the even cells. Imagine if these modules are
actually the same, but if checkers used one seed for all the odd
cells and another seed for all the even cells. If the modules
used actually utilized the seed provided, checkers would be able
to differentiate the odd from even by seeding them differently
even when the modules are the same.
This commit is a step in that direction, but rototiller and all
the composite modules (rtv,compose,montage) are simply passing
rand() as the seeds. Also none of the modules have yet been
modified to actually make use of these seeds.
Subsequent commits will update modules to seed their
pseudo-randomized initial state from the seed value rather than
always calling things like rand() themselves.
|
|
Most of the threaded modules have settled down on two basic forms
of fragmenter function:
1. a slice per cpu, where tile-oriented locality isn't useful
2. ~64x64 tiles, in scenarios where screen-space locality helps
Now that n_cpus is wired up to the fragmenter, #1 can be
fulfilled without requiring a module-private context plumbing
n_cpus from create_context().
A future commit will replace some module-specific fragmenters by
returning one of these instead as res_fragmenter in their
prepare_frame(), wherever applicable.
|
|
Fragmenting is often dimensioned according to the number of cpus,
and by not supplying this to the fragmenter it was made rather
common for module contexts to plumb this themselves - in some
cases incorporating a context type/create/destroy rigamarole
for the n_cpus circuit alone.
So just plumb it in libtil, and the prepare_frame functions can
choose to ignore it if they have something more desirable onhand.
Future commits will remove a bunch of n_cpus from module contexts
in favor of this.
|
|
For modules which are overlay-appropriate, they should indicate
it when initializing their respective til_module_t. If they
intend to participate in automagic compositing as overlays
anyways.
|
|
This brings something resembling an actual type to the private
objects returrned in *res_setup. Internally libtil/rototiller
wants this to be a til_setup_t, and it's up to the private users
of what's returned in *res_setup to embed this appropriately and
either use container_of() or casting when simply embedded at the
start to go between til_setup_t and their private containing
struct.
Everywhere *res_setup was previously allocated using calloc() is
now using til_setup_new() with a free_func, which til_setup_new()
will initialize appropriately. There's still some remaining work
to do with the supplied free_func in some modules, where free()
isn't quite appropriate.
Setup freeing isn't actually being performed yet, but this sets
the foundation for that to happen in a subsequent commit that
cleans up the setup leaks.
Many modules use a static default setup for when no setup has
been provided. In those cases, the free_func would be NULL,
which til_setup_new() refuses to do. When setup freeing actually
starts happening, it'll simply skip freeing when
til_setup_t.free_func is NULL.
|
|
This makes the arg return optional by using a res_arg pointer,
instead returning -ENOMEM when it would have returned NULL on
allocation failures.
This also makes it possible to detect when no setup was
performed, by returning 0 in such a case. Now returns 1 when
setup occurs and res pointers populated.
|
|
This commit pulls the setup randomizer out of rtv into libtil
proper, so other modules may make use of it.
Other than adding an assert no functional changes occurred.
It may make sense to split this into two functions; one which
takes a til_module_t as-is, and a lower-level bare setup function
callback based function that doesn't know about til_module_t the
former would call into. That way generic setup randomization can
occur (the same setup machinery is used in video contexts for
example) without necessarily having a til_module_t on hand.
|
|
This is a preparatory commit for cleaning up the existing sloppy
global-ish application of settings during the iterative _setup()
call sequences.
Due to how this has evolved from a very rudimentary thing
enjoying many assumptions about there ever only being a single
module instance being configured by the settings, there's a lot
of weirdness and inconsistency surrounding module setup WRT
changes being applied instantaneously to /all/ existing and
future context's renderings of a given module vs. requiring a new
context be created to realize changes.
This commit doesn't actually change any of that, but puts the
plumbing in place for the setup methods to allocate and
initialize a private struct encapsulating the parsed and
validated setup once the settings are complete. This opaque
setup pointer will then be provided to the associated
create_context() method as the setup pointer. Then the created
context can configure itself using the provided setup when
non-NULL, or simply use defaults when NULL.
A future commit will update the setup methods to allocate and
populate their respective setup structs, adding the structs as
needed, as well as updating their create_context() methods to
utilize those setups.
One consequence of these changes when fully realized will be that
every setting change will require a new context be created from
the changed settings for the change to be realized.
For settings appropriately manipulated at runtime the concept of
knobs was introduced but never finished. That will have to be
finished in the future to enable more immediate/interactive
changing of settings-like values appropriate for interactive
manipulation
|
|
Now that til_setting_t.desc is not only a thing, but a thing that
is intended to be refreshed regularly in the course of things
like GUI interactive settings construction, it's not really
appropriate to try even act like this these are const anymore.
|
|
|
|
Originally the thinking was that rototiller modules would become
dlopen()ed shared objects, and that it would make sense to let
them be licensed differently.
At this time only some modules I have written were gplv3, Phil's
modules are all gplv2, and I'm not inclined to pivot towards a
dlopen model.
So this commit drops the license field from til_module_t,
relicenses my v3 code to v2, and adds a gplv2 LICENSE file to the
source root dir. As of now rototiller+libtil and all its modules
are simply gplv2, and anything linking in libtil must use a gplv2
compatible license - the expectation is that you just use gplv2.
|
|
The existing iterative *_setup() interface only described
settings not found, quietly accepting usable settings already
present in the til_settings_t.
This worked fine for the existing interactive text setup thing,
but it's especially problematic for providing a GUI setup
frontend.
This commit makes it so the *_setup() methods always describe
undescribed settings they recognize, leaving the setup frontend
loop calling into the *_setup() methods to both apply the
description validation if wanted and actually tie the description
to respective setting returned by the _setup() methods as being
related to the returned description.
A new helper called til_settings_get_and_describe_value() has
been introduced primarily for use of module setup methods to
simplify this nonsense, replacing the til_settings_get_value()
calls and surrounding logic, but retaining the til_setting_desc_t
definitions largely verbatim.
This also results in discarding of some ad-hoc
til_setting_desc_check() calls, now that there's a centralized
place where settings become "described" (setup_interactively in
the case of rototiller).
Now a GUI frontend (like glimmer) would just provide its own
setup_interactively() equivalent for constructing its widgets for
a given *_setup() method's chain of returned descs. Whereas in
the past this wasn't really feasible unless there was never going
to be pre-supplied settings.
I suspect the til_setting_desc_check() integration into
setup_interactively() needs more work, but I think this is good
enough for now and I'm out of spare time for the moment.
|
|
Largely mechanical rename of librototiller -> libtil, but
introducing a til_ prefix to all librototiller (now libtil)
functions and types where a rototiller prefix was absent.
This is just a step towards a more libized librototiller, and til
is just a nicer to type/read prefix than rototiller_.
|
