Development

Changes in Ansel packages distribution

Development

Target clones

Compiling the software natively on the computer use to improve runtimes on CPU by around 30 %, compared to pre-built packages. The reason is the compiler makes specific optimizations for the target hardware on which it is compiled, while pre-built packages have to stay generic and trigger more conservative optimization for the sake of wide support. Note that OpenCL kernels are, anyway, compiled for your particular GPU using your OpenCL driver, so the story is different there.

Farbregler endlich korrekt

Development Design

Einführung

Werkzeuge vs. Maschinen, Handwerk vs. Industrie

In seinem Buch The Technological Society (1954) stellt Jacques Ellul den Unterschied zwischen der vor- und der nachindustriellen Revolution wie folgt dar:

Die vorindustrielle Ära ist die Herrschaft des Werkzeugs und des Handwerks. Die vornehmste Eigenschaft von Werkzeugen ist es, generisch, vielseitig und anpassungsfähig zu sein. Es obliegt dem Handwerker, seine Fähigkeiten zu entwickeln, um die Werkzeuge seiner Absicht folgen zu lassen, sodass die Hand die Grenzen des Instruments ausgleicht. Dieses Konzept ist Musikern noch heute wohlbekannt: Man muss üben, lernen, versuchen, scheitern, erneut versuchen … es gibt keine Abkürzungen. Ellul betont den Gedanken der Sparsamkeit, der mit Werkzeugen einhergeht: Ressourcen sind begrenzt, sodass Ihr Werkzeugbestand im Grunde dadurch bestimmt ist, was Sie sich leisten, tragen, meistern und vor Ort bauen können. Trends ändern sich langsam und sind lokal, weil sie lokale Ressourcen nutzen und sich an lokale Bedürfnisse anpassen, und Werkzeuge folgen demselben Muster. Werkzeuge sind Erbstücke, die vom Meister an den Lehrling, von den Eltern an das Kind weitergegeben werden. Sie werden nicht inkompatibel oder veraltet.

Redesigning the lighttable and the mipmap cache

Development Redesign Performance

Between January 2022 and March 2026, Ansel landed 297 non-merge commits regarding the lighttable grid, its thumbnails, and their rendering pipeline and caching. I tried to make do with the scruffy Darktable lighttable code, only degreased, for as long as I could but unfortunately, it was pure technical debt and it was painfully slow.

Indeed, Darktable “manages” the crappiness of its lighttable by reducing its size: the left and right side panels take a lot of display surface, which leaves even less area for the lighttable to repaint. Since Ansel removed the right side panel, merging its content with the left one and the global menu, there was more surface to paint, more CPU work to do, and the terrible design of the lighttable became all the more harmful.

Komplette Überarbeitung der Pipeline

Development

Seit ich um 2012 herum begann, Darktable zu benutzen, war ich immer überrascht, wie wenig RAM es verwendete. Die Leute halten es für eine gute Sache, wenn eine Anwendung sparsam mit dem Speicher umgeht, und das trifft sicherlich zu, wenn wir von Ihrer Desktop-Umgebung sprechen. Aber wenn wir von einer Produktionssoftware sprechen, die aufwendiges Pixel-Rendering an Bildern von 12 bis 54 Mpixel durchführt, dann bedeutet das, dass dieselben aufwendigen Berechnungen immer wieder durchgeführt werden, anstatt gespeichert zu werden, um später wiederverwendet zu werden. Genau dafür ist ein Cache da: um teure Berechnungen zu vermeiden. Und er sollte dafür den gesamten verfügbaren RAM nutzen, denn Rechnen bedeutet Energieverschwendung, und das hat konkrete Auswirkungen, wenn Sie im Akkubetrieb arbeiten. Außerdem haben Sie für diesen RAM bezahlt, und ihn zu nutzen leert Ihren Akku nicht. Die CPU/GPU hingegen…

Verlaufszusammenführung robust und vorhersehbar gemacht

Development

Memoiren eines Typen, der zu viel Zeit damit verbracht hat, den Mist anderer Leute wegzuräumen und für ihre schlechten Entscheidungen zu bezahlen, Folge #zu viele.

Verlauf und Stile per Kopieren/Einfügen zu übertragen, sind Kernfunktionen in Ansel und diejenige, die ihm den Titel „Workflow-App" verdient (oder auch nicht). Aber es ist auch eine der intern am schwersten korrekt umzusetzenden. Nutzer sehen eine Liste von Bearbeitungen, doch unter der Haube hängen diese Bearbeitungen von der Reihenfolge der Pipeline, den Modulinstanzen und den Masken ab. Wenn zwei Bilder unterschiedliche Pipeline-Topologien haben, kann naives Kopieren von Bearbeitungen zu Inkonsistenzen führen.

Welcome, developer documentation !

Development

Back in December 2019, I asked that someone took care of providing AppImages packages  for Darktable. The obvious benefit would have been enabling early testing, prior to release, from people who can’t build the source code themselves, as to hopefully provide early feedback and help debugging before releasing. This has never been a priority, which means that it was ok to have a pre-release and a post-release rush to fix bugs.

Fixing the pipeline cache and 10 years-old bugs

Development

Recap of the previous episodes

  1. Between 2020 and 2022, Darktable underwent a mass-destruction enterprise, by a handful of guys with more freetime and benevolence than actual skills,
  2. In 2022, I started noticing an annoying lag  between GUI interactions with sliders controls and feedback/update of said sliders. For lack of feedback stating that the value change was recorded, users could change it again, thereby starting additionnal pipeline recomputes and effectively freezing their computer because stupid GUI never said “got you, wait for a bit now”.
  3. I discovered that pipeline recomputations orders were issued twice per click (once on “button pushed”, once on “button released” events), and once again for each mouse motion, but also that the GUI states were updated seemingly after pipe recompute.
  4. I fixed that by almost rewriting the custom GUI controls (Bauhaus lib). I thought that preventing reckless recompute orders was gonna solve the lag : it didn’t. Then, I discovered that requesting a new pipeline recompute before the previous ended waited for the previous to end, despite a shutdown mechanism implemented many years ago that should have worked.
  5. I fixed that by implementing a kill-switch mechanism on pipelines, following comments in the code from the 2010’s and internal utilities that may well have never worked. This did not always work because the kill order came often with a noticeable delay. Once again, the GUI lag was not fixed.

Rewriting the import tool

Development Redesign

Ansel inherits from Darktable its database backbone: the non-destructive editing histories are saved per-picture into an SQLite database, along with metadata and other user-defined data. Making the database aware of new pictures is done through “importing” pictures from a disk or a memory card. That’s where the import tool comes.

Unfortunately, the Darktable importer is another thing that was butchered circa 2020 and turned into something deeply disconcerning, as it is a file browser that resembles no previously-known file browser, and manages to lack basic features (like Ctrl+F or EXIF preview) while still being bloated with useless ones (see below). This is where we loose many a future user, and it is only step 0 of the workflow. What a great showcase of what a “workflow app” can do !

Implementing kill-switch on pipeline

Development

I have thought, for a very long time, that there was some kill-switch mechanism on the pixel pipeline. The use case is the following :

  1. you are changing a module parameter,
  2. the previews (the central darkroom one and the thumbnail in left panel, also used for histogram and color pickers) recompute their pipeline to account for that change,
  3. one of the previews finishes rendering before the other, and the result is obviously not what you wanted,
  4. you change again the module parameter, without waiting for the recomputation to finish.

In that case, you want to kill all active pipelines because their output will not be used, and start recomputing everything immediately with new parameters. Except Darktable doesn’t do that, it lets the pipeline finish before restarting it, and looking at the comments in the source code, it seems to be a fairly recent regression and not the originally intended behaviour.

Un-darktable-ing GUI controls

Development

Darktable has its own GUI widgets library, for sliders and comboboxes (aka drop-down menus or selection boxes), called Bauhaus (in the source code, it’s in src/bauhaus/bauhaus.c). While they use Gtk as a backend, Bauhaus are custom objects. And like many things in Darktable, custom equals rotten.

In 2022, ‍I noticed parasite redrawings and lags , when using them, leading to a frustrating user experience : the widget redrawing seemed to wait for pipeline recomputations to complete, which meant that users were not really sure their value change was recorded, which could lead them to try again, starting another cycle of expensive recomputation, and effectively freezing their computer for several very frustrating minutes of useless intermediate pipeline recomputations.

New build options for Linux

Development

I accidentally discovered that the Linux build script used a “package” build, meaning the CPU optimizations are limited to generic ones in order to produce portable binaries that can be installed on any x86-64 platform. By “using”, I mean the package build was not explicitely disabled, so it was enabled by default.

Anyway, this is now disabled by default, since the actual packages (.exe and .appimage) are not built through that script, which is primarily meant to help end-users. To get the previous behaviour back, you would need to run:

Dev diary #2 : introducing Chantal

Development

2022 was so bad in terms of junk emails and noise that I started the Virtual Secretary , a Python framework to write intelligent email filters by crossing information between several sources to guess what incoming emails are and whether they are important/urgent or not. When I’m talking about junk emails, it’s also Github notifications, pings on pixls.us (thank God I closed my account on that stupid forum), YouTube, and direct emails from people hoping to get some help in private.

Dev diary

Development

It’s been roughly 3 months that I rebranded “R&Darktable” (that nobody seemed to get right), into “Ansel”, then bought the domain name and created the website from scratch with Hugo (I had never programmed in Golang before, but it’s mostly template code).

Then I spent a total 70 h on making the nightly packages builds for Windows and Linux work for continuous delivery, something that Darktable never got right (“you can build yourself, it’s not difficult”), only to see the bug tracker blow up after release (nothing better than chaining the pre-release sprint with a post-release one to reduce your life expectancy).

Search

You can also ask Chantal, the AI search engine.