Development

Changes in Ansel packages distribution

Development

Target clones

Compiling the software natively on the computer use to improve runtimes on CPU by around 30 %, compared to pre-built packages. The reason is the compiler makes specific optimizations for the target hardware on which it is compiled, while pre-built packages have to stay generic and trigger more conservative optimization for the sake of wide support. Note that OpenCL kernels are, anyway, compiled for your particular GPU using your OpenCL driver, so the story is different there.

Los controles de color por fin correctos

Development Design

Introducción

Herramientas frente a máquinas, artesanía frente a industria

En su libro The Technological Society (1954), Jacques Ellul presenta la diferencia entre las épocas anterior y posterior a la revolución industrial de la siguiente manera:

La era preindustrial es el reinado de la herramienta y de la artesanía. La propiedad primordial de las herramientas es ser genéricas, versátiles y adaptables. Corresponde al artesano desarrollar sus habilidades para hacer que las herramientas sigan su intención, de modo que la mano supla las limitaciones del instrumento. Este concepto sigue siendo bien conocido hoy por los músicos: tienes que practicar, aprender, intentar, fallar, reintentar… no hay atajos. Ellul insiste en la idea de parsimonia que acompaña a las herramientas: los recursos son limitados, así que tu conjunto de herramientas queda prácticamente definido por lo que puedes permitirte, transportar, dominar y construir localmente. Las modas cambian despacio y son locales, porque usan recursos locales y se adaptan a necesidades locales, y las herramientas siguen el mismo patrón. Las herramientas son legados que pasan de maestro a aprendiz, de padre a hijo. No se vuelven incompatibles ni obsoletas.

Redesigning the lighttable and the mipmap cache

Development Redesign Performance

Between January 2022 and March 2026, Ansel landed 297 non-merge commits regarding the lighttable grid, its thumbnails, and their rendering pipeline and caching. I tried to make do with the scruffy Darktable lighttable code, only degreased, for as long as I could but unfortunately, it was pure technical debt and it was painfully slow.

Indeed, Darktable “manages” the crappiness of its lighttable by reducing its size: the left and right side panels take a lot of display surface, which leaves even less area for the lighttable to repaint. Since Ansel removed the right side panel, merging its content with the left one and the global menu, there was more surface to paint, more CPU work to do, and the terrible design of the lighttable became all the more harmful.

Revisión completa del pipeline

Development

Desde que empecé a usar Darktable, allá por 2012, siempre me sorprendió lo poca RAM que usaba. La gente cree que es bueno que una aplicación use la memoria con moderación, y eso es cierto sin duda si hablamos de tu entorno de escritorio. Pero hablando de un software de producción que hace un renderizado intensivo de píxeles sobre imágenes de 12 a 54 Mpíxeles, esto significa que los mismos cálculos pesados se hacen una y otra vez en lugar de guardarse para reutilizarse más tarde. Para eso sirve una caché: para evitar cálculos costosos. Y para eso debería usar toda la RAM disponible, porque calcular es malgastar energía, y eso tiene un impacto concreto si trabajas con batería. Además, pagaste por esa RAM y usarla no vacía tu batería. La CPU/GPU, por otro lado…

La fusión de historiales, ahora robusta y predecible

Development

Memorias de un tipo que pasó demasiado tiempo limpiando la mierda de los demás y pagando por sus malas decisiones, episodio #demasiados.

Copiar/pegar historiales y estilos son funciones fundamentales en Ansel, y las que hacen que merezca (o no) su título de «aplicación de flujo de trabajo». Pero también están entre las más difíciles de acertar internamente. Los usuarios ven una lista de ediciones, pero por debajo esas ediciones dependen del orden del pipeline, de las instancias de módulos y de las máscaras. Si dos imágenes tienen topologías de pipeline distintas, copiar ediciones de forma ingenua puede producir inconsistencias.

Welcome, developer documentation !

Development

Back in December 2019, I asked that someone took care of providing AppImages packages  for Darktable. The obvious benefit would have been enabling early testing, prior to release, from people who can’t build the source code themselves, as to hopefully provide early feedback and help debugging before releasing. This has never been a priority, which means that it was ok to have a pre-release and a post-release rush to fix bugs.

Fixing the pipeline cache and 10 years-old bugs

Development

Recap of the previous episodes

  1. Between 2020 and 2022, Darktable underwent a mass-destruction enterprise, by a handful of guys with more freetime and benevolence than actual skills,
  2. In 2022, I started noticing an annoying lag  between GUI interactions with sliders controls and feedback/update of said sliders. For lack of feedback stating that the value change was recorded, users could change it again, thereby starting additionnal pipeline recomputes and effectively freezing their computer because stupid GUI never said “got you, wait for a bit now”.
  3. I discovered that pipeline recomputations orders were issued twice per click (once on “button pushed”, once on “button released” events), and once again for each mouse motion, but also that the GUI states were updated seemingly after pipe recompute.
  4. I fixed that by almost rewriting the custom GUI controls (Bauhaus lib). I thought that preventing reckless recompute orders was gonna solve the lag : it didn’t. Then, I discovered that requesting a new pipeline recompute before the previous ended waited for the previous to end, despite a shutdown mechanism implemented many years ago that should have worked.
  5. I fixed that by implementing a kill-switch mechanism on pipelines, following comments in the code from the 2010’s and internal utilities that may well have never worked. This did not always work because the kill order came often with a noticeable delay. Once again, the GUI lag was not fixed.

Rewriting the import tool

Development Redesign

Ansel inherits from Darktable its database backbone: the non-destructive editing histories are saved per-picture into an SQLite database, along with metadata and other user-defined data. Making the database aware of new pictures is done through “importing” pictures from a disk or a memory card. That’s where the import tool comes.

Unfortunately, the Darktable importer is another thing that was butchered circa 2020 and turned into something deeply disconcerning, as it is a file browser that resembles no previously-known file browser, and manages to lack basic features (like Ctrl+F or EXIF preview) while still being bloated with useless ones (see below). This is where we loose many a future user, and it is only step 0 of the workflow. What a great showcase of what a “workflow app” can do !

Implementing kill-switch on pipeline

Development

I have thought, for a very long time, that there was some kill-switch mechanism on the pixel pipeline. The use case is the following :

  1. you are changing a module parameter,
  2. the previews (the central darkroom one and the thumbnail in left panel, also used for histogram and color pickers) recompute their pipeline to account for that change,
  3. one of the previews finishes rendering before the other, and the result is obviously not what you wanted,
  4. you change again the module parameter, without waiting for the recomputation to finish.

In that case, you want to kill all active pipelines because their output will not be used, and start recomputing everything immediately with new parameters. Except Darktable doesn’t do that, it lets the pipeline finish before restarting it, and looking at the comments in the source code, it seems to be a fairly recent regression and not the originally intended behaviour.

Un-darktable-ing GUI controls

Development

Darktable has its own GUI widgets library, for sliders and comboboxes (aka drop-down menus or selection boxes), called Bauhaus (in the source code, it’s in src/bauhaus/bauhaus.c). While they use Gtk as a backend, Bauhaus are custom objects. And like many things in Darktable, custom equals rotten.

In 2022, ‍I noticed parasite redrawings and lags , when using them, leading to a frustrating user experience : the widget redrawing seemed to wait for pipeline recomputations to complete, which meant that users were not really sure their value change was recorded, which could lead them to try again, starting another cycle of expensive recomputation, and effectively freezing their computer for several very frustrating minutes of useless intermediate pipeline recomputations.

New build options for Linux

Development

I accidentally discovered that the Linux build script used a “package” build, meaning the CPU optimizations are limited to generic ones in order to produce portable binaries that can be installed on any x86-64 platform. By “using”, I mean the package build was not explicitely disabled, so it was enabled by default.

Anyway, this is now disabled by default, since the actual packages (.exe and .appimage) are not built through that script, which is primarily meant to help end-users. To get the previous behaviour back, you would need to run:

Dev diary #2 : introducing Chantal

Development

2022 was so bad in terms of junk emails and noise that I started the Virtual Secretary , a Python framework to write intelligent email filters by crossing information between several sources to guess what incoming emails are and whether they are important/urgent or not. When I’m talking about junk emails, it’s also Github notifications, pings on pixls.us (thank God I closed my account on that stupid forum), YouTube, and direct emails from people hoping to get some help in private.

Dev diary

Development

It’s been roughly 3 months that I rebranded “R&Darktable” (that nobody seemed to get right), into “Ansel”, then bought the domain name and created the website from scratch with Hugo (I had never programmed in Golang before, but it’s mostly template code).

Then I spent a total 70 h on making the nightly packages builds for Windows and Linux work for continuous delivery, something that Darktable never got right (“you can build yourself, it’s not difficult”), only to see the bug tracker blow up after release (nothing better than chaining the pre-release sprint with a post-release one to reduce your life expectancy).

Search

You can also ask Chantal, the AI search engine.