Announcing DNF 3 development

Posted by Daniel Mach , Jaroslav Mracek on March 22, 2018
We are pleased to announce that development of DNF 3 has started. This version focused on performance improvements, new API and consolidating the whole software management stack. That's why the rest of the text is more about libdnf than DNF, which should become smaller and only wrap underlying libdnf library.

What's going on with libdnf?

If you are watching DNF and libdnf development, you may ask what's going on lately. There is a lot of activity in libdnf GIT. Let us share our plans with you. Originally there were two libraries - hawkey providing query and dependency resolution interface to DNF and libhif as a PackageKit backend. These two merged into libdnf, but the merge hasn't been fully finished. The technical debt grows as new code is being added and there's even disparity between libdnf and DNF. We decided to change this and make libdnf sane again.

Improvements made so far

We made several improvements to queries already. They are executed as part of all DNF commands and have significant performance impact. We focused mainly on security queries and searching packages by Name or full NEVRA.
Let's look at some numbers. The chart shows execution times (lower is better) of several DNF commands using 3 different versions of libdnf:
  • 0.9.1 - initial version in Fedora 26
  • 0.11.1 - latest released version in Fedora 26 and 27
  • 0.13.0 - development version in GIT

What we want to achieve

  • Performance improvement
  • Consistent behavior across the whole software management stack
    • Move existing business logic from DNF to libdnf
    • Remove duplicate code
  • Well-defined and maintainable API
  • New high-level functionality (like “dnf.Base()”)
  • Remain compatible with existing APIs (C/Python) as much as possible

How to move forward

We decided to move libdnf from C to C++ for several good reasons:
  • C++ is a standardized object-oriented language with good compiler support
  • Existing C code compiles with C++ compiler - we can reuse some existing code as is
  • C++ 11 is supported by GCC in RHEL 7 / CentOS 7
  • C++ has more "batteries included" than C (classes, vectors and maps, shared pointers, algorithm library, etc.)
  • You don't have to worry about raw pointers in C++.
  • SWIG bindings (C++ objects wrapped in shared pointer naturally bind to Python objects)

Road-map

Doing all the consolidation, we have a unique opportunity to redesign libdnf. This is what we are planning:
  1. Preparation (mostly done)
    • Remove unused functionality
    • De-duplicate some code between DNF and libdnf
  2. DNF Integration (in progress)
    • Design new functionality in libdnf, and replace corresponding DNF code in Python
  3. Redesign libdnf
    • Split libdnf into subsystems
    • Provide new high-level interface (replacing dnf.Base and libdnf context)
  4. Broader integration
    • Use the new libdnf API in PackageKit, rpm-ostree and microdnf

Benefits

You may ask: "How will I benefit from all these changes?" First of all, you will get consistent user experience from all software management tools. Wouldn't it be great if PackageKit and DNF shared repository and package cache? And that's just a tip of the iceberg. You will also get much better performance, because the DNF team has been analyzing the whole stack. Asking "why" often results in removing bottlenecks all over the place.