*fb: improve error propagation out of setup/init

A lot of errors were being conflated as ENOMEM due to the lazy
use of NULL pointer returns for errors.

This commit reworks a handful of those return paths to instead
return an errno-style int, storing the results on success at a
supplied result pointer.

It's kind of ugly, and I make some assumptions about libdrm
setting errno on failure - it too uses this lazy API of returning
NULL pointers on failure.  Hopefully errno is always set by an
underlying ioctl failing.

The SDL error API is also pretty gross, being cross-platform it
defines its own error codes so I try vaguely map these to errno
values.

I'm considering this a first approximation at fixing this up, but
there are probably bugs as I did it real fast and nasty.

It at least seems to all still work OK here in the non-error
paths I tested.  So it doesn't seem more broken than before at a
glance.

1 files changed, 2 insertions, 2 deletions

diff --git a/src/fb.h b/src/fb.h
index d9914df..b06ecb8 100644
--- a/src/fb.h
+++ b/src/fb.h
@@ -34,7 +34,7 @@ typedef struct fb_page_t {
 /* Supply this struct to fb_new() with the appropriate context */
 typedef struct fb_ops_t {
 	int	(*setup)(const settings_t *settings, setting_desc_t **next);
-	void *	(*init)(const settings_t *settings);
+	int	(*init)(const settings_t *settings, void **res_context);
 	void	(*shutdown)(void *context);
 	int	(*acquire)(void *context, void *page);
 	void	(*release)(void *context);
@@ -49,7 +49,7 @@ fb_page_t * fb_page_get(fb_t *fb);
 void fb_page_put(fb_t *fb, fb_page_t *page);
 void fb_free(fb_t *fb);
 void fb_get_put_pages_count(fb_t *fb, unsigned *count);
-fb_t * fb_new(const fb_ops_t *ops, settings_t *settings, int n_pages);
+int fb_new(const fb_ops_t *ops, settings_t *settings, int n_pages, fb_t **res_fb);
 int fb_flip(fb_t *fb);
 void fb_fragment_divide(fb_fragment_t *fragment, unsigned n_fragments, fb_fragment_t fragments[]);
 int fb_fragment_slice_single(const fb_fragment_t *fragment, unsigned n_fragments, unsigned num, fb_fragment_t *res_fragment);