Tuesday, January 31, 2017

SSSD: {DBus,Socket}-activated responders!

Since its 1.15.0 release, SSSD takes advantage of systemd machinery and introduces a new way to deal with the responders.

Previously, in order to have a responder initialized, the admin would have to add the specific responder to the "services" line in sssd.conf file, which does make sense for the responders that are often used but not for those rarely used (as the infopipe and PAC responders for instance).

This old way is still preserved (at least for now) and this new release is fully backwards-compatible with the old config file.

For this new release, however, adding responders to the "services" isn't needed anymore as the admin can easily enable any of the responders' sockets and those will be {dbus,socket}-activated on demand and will be up while are still being used. In case the responder becomes idle, it will automatically shut itself down after a configurable amount of time.

The sockets we've created are: sssd-autofs.socket, sssd-nss.socket, sssd-pac.socket, sssd-pam.socket (and sssd-pam-priv.socket, but you don't have to worry about this one), sssd-ssh.socket and sssd-sudo.socket. As an example, considering the admins want to enable the sockets for both NSS and PAM responders, they should do: `systemctl enable sssd-pam.socket sssd-nss.socket` and voilà!

In some cases the admins may also want to set the "responder_idle_timeout" option added for each of the responders in order to tweak for how long the responder will be running in case itbecomes idle. Setting this option to 0 (zero) disables the responder_idle_timeout. For more details, please, check sssd.conf man page.

For this release we've taken a more conservative path and are leaving up to the admins to enable the services they want to have enabled in case they would like to try to using {dbus,socket}-activated responders

It's also important to note that while the SELinux policies are not updated in your distro you may need to have SELinux in permissive mode in order to test/use the {dbus,socket}-activated responders. A bug for this is already filed for Fedora and hopefully will be fixed before the new package is included in the distro.

And the changes in the code were (a high-level explanation) ...

Before this work the monitor was the piece of code responsible for handling the responders listed in the services' line of sssd.conf file. And by handling I mean:

  • Gets the list of services to be started (and, consequently, the total number of services);
  • For each service:
    • Gets the service configuration;
    • Starts the service;
    • Adds the service to the services' list;
    • Once the service is up, a dbus message is sent to the monitor, which ...
      • Sets up the sbus* connection to communicate with the service;
      • Marks the service as started;

Now, the monitor does (considering an empty services' line):

  • Once the service is up, a dbus message is sent to the monitor;
    • The number of services is increased;
    • Gets the service configuration;
    • Adds the service to the services' list
    • Sets up the sbus connection to communicate with the service;
    • Sets up a destructor to the sbus connection in order to properly shutdown the service when this connection is closed;
    • Marks the service as started;

By looking at those two different processes done by the monitor, some of you may have realized an extra step needed when the service has been {dbus,socket}-activated that was needed at all before. Yep, "Sets up a destructor to the sbus connection in order to properly shutdown the service when this connection is closed" is a completely new thing as, previously, the services were just shut down when SSSD was shut down and now the services are shutdown when they become idle.

So, what's basically done now is:
 - Once there's no communication to the service, it's (sbus) connection with the monitor is closed;
 - Closing the (sbus) connection triggers the following actions:
    - The number of services is decreased;
    - The connection destructor is unset (otherwise it would be called again on the service has been freed);
    - Service is shut down:

*sbus: SSSD uses dbus protocol over a private socket to handle its internal communication, so the services do not talk over system bus.

And how do the unit files look like?

SSSD has 7 services: autofs, ifp, nss, pac, pam, ssh and sudo. From those 7 services 4 of them have pretty much these unit files:

AutoFS, PAC, SSH and Sudo unit files:

Description=SSSD $(responder) Service responder


ExecStartPre=-/bin/chown $sssd_user:$sssd_user /var/log/sssd/sssd_autofs.log
ExecStart=/usr/libexec/sssd/sssd_$responder --debug-to-files --socket-activated


Description=SSSD $(responder) Service responder socket



And about the different ones? We will get there ... and also explain why they are different.

The infopipe (ifp) unit file:

As the infopipe won't be socket-activated, it doesn't have the its respective .socket unit.
Also, differently than the others responders the infopipe responder can only be run as root nowadays.
In the end, its .service unit looks like:

Description=SSSD IFP Service responder

ExecStart=/usr/libexec/sssd/sssd_ifp --uid 0 --gid 0 --debug-to-files --dbus-activated

The PAM unit files:

The main difference between PAM responder and the others is that PAM has two sockets that can end up socket-activating its service. Also, these sockets have a special permission.
In the end, its unit files look like:

Description=SSSD PAM Service responder

Also=sssd-pam.socket sssd-pam-priv.socket

ExecStartPre=-/bin/chown $sssd_user:$sssd_user @logpath@/sssd_pam.log
ExecStart=@libexecdir@/sssd/sssd_pam --debug-to-files --socket-activated

Description=SSSD PAM Service responder socket



Description=SSSD PAM Service responder private socket



The NSS unit files:

The NSS responder was the trickiest one to have working properly, mainly because when socket-activated it has to run as root.
The reason behind this is that systemd calls getpwnam() and getgrnam() when using "User="/"Group=" different than root. By doing this libc ends up querying for $sssd_user, trying to talk to NSS responder which is not up yet and then the clients would end up hanging for a few minutes (due to our default_client_timeout) which is something we really want to avoid.

In the end, its unit files look like:

Description=SSSD NSS Service responder


ExecStartPre=-/bin/chown root:root @logpath@/sssd_nss.log
ExecStart=@libexecdir@/sssd/sssd_nss --debug-to-files --socket-activated

Description=SSSD NSS Service responder socket


All the services' units have a "BindsTo=sssd.service" in order to ensure that the service will be stopped when sssd.service is stopped so in case SSSD is shutdown/restart those actions will be propagated to the responders as well.

Similarly to "BindsTo=ssssd.service" there's "WantedBy=sssd.service" in every socket unit and it's there to ensure that, once the socket is enabled it will be automatically started by SSSD when SSSD is started.

And that's pretty much all changes that I've covered with this work.

I really have to say a big thank you to ...

  • Lukas Nykryn and Michal Sekletar who patiently reviewed the unit files we're using and gave me a lot if good tips while doing this work;
  • Sumit Bose who helped me to find out the issue with the NSS responder when trying to run it as a non-privileged user;
  • Jakub Hrozek, Lukas Slebodnik and Pavel Brezina for reviewing and helping me to find bugs, crashes, regressions that fortunately were avoided.

And what's next?

There's already a patch making the {dbus,socket}-activated automatically enabled when SSSD starts, which changes our approach from having to explicit enable the sockets in order to take advantage of this work to explicitly mask the disable (actually, mask) the sockets of the processes that shouldn't be {dbus,socket}-activated.

Also, a bigger work for the future is to also have the providers being socket-activated, but this is material for a different blob post. ;-)

Nice, nice. But I'm having issues with what you've described!

In case it happens to you, please, keep in mind that the referred way to diagnose any issues would be:

  • Inspecting sssd.conf in order to check which are the explicitly activated responders in the services' line;
  • `systemctl status sssd.service`;
  • `systemctl status sssd-$responder.service` (for the {dbus,socket}-activated ones);
  • `journalctl -u sssd.service`;
  • `journalctl -u sssd-$responder.service` (for the {dbus,socket}-activated ones);
  • `journalctl -br`;
  • Checking SSSD debug logs in order to see whether SSSD sockets where communicated

Sunday, December 25, 2016


It's been quite a long time I don't write anything here and the main reason is that I've been quite busy lately. But, well, as the end of the year has been approaching quite faster than expected I guess this is the right time to, at least, sum up what happened during the last year or so.

2016 hasn't been exactly what I'd call the "best year of my life" but some important stuff have happened and are worth to mention ...

Pain! A lot of pain!
By the beginning of the year I had a few back problems. They were not exactly related to each other, but all of them related to overweight. :-\.

So, in February I had a pilonidal cyst, which came back a few times over the year just to remind me to update my definition of pain. It's something quite simple to treat, but the pain can be really horrible
There's a surgery that can be done in order to (try to) fix the root cause of the issue but the recovering process seems way less appealing than treating it a few times per year when needed.

Apart from this issue, I've started feeling some pain on my lower-back that, after a few days, went all the way down to my knee and for a few weeks limited my movements. Finding the root-cause of this issue was not so easy, but thanks to Pavel Grunt's mother I ended up visiting a neurologist who found it out: a slipped disc!
Again, there's a surgery that can be done and, again, I decided to try to work it around.

Time to change?
I clearly remember a friend of mine who told you "(...) at some point you'll have to re-think your habits (...)". Seems that the time has arrived and there were a lot of things that weren't making me exactly happy about my life. So, I've decided to give a try and change two of those.

So long, SPICE!
I've worked on SPICE for circa 2 years, mainly working on the client-side (spice-gtk and virt-viewer) for both Windows and Linux. An experience that I, sincerely, can't complain.

Both the project and the people there are amazing but I wasn't feeling as excited as I knew I could feel. So, with the heart-broken and quite uncertain about the future, I've decided to leave the project. I've decided to search for something else and have a completely new thing to occupy my mind during 8+ hours/day.

Nice to meet you, SSSD!
SSSD was my primary target and choice.  Why? Well, I'm not sure I could answer this question by that time. I'm still not sure I could answer this question right now.
What I can tell, for sure, is that I feel more and more like I've chosen wisely.

Work-wise the change has been quite satisfactory. I'm spending way more time writing code than I was when working for the SPICE team. As pretty much everything related to Identity Management is new to me, I'm (naturally) spending way more time learning stuff than I was during the last year working for the SPICE team.

Adaptation period
It's really important to mention that the adaptation period was not as smooth as I thought it would be. I moved from a completely international team to a team where I'm the only "foreigner" (well, the team is a mix of Czechs and Slovaks, but they speak their local language when talking to each other) and it was a big culture shock as, due to the language barrier, I wasn't able to feel part of the team, I wasn't able to feel comfortable to interact with the team and it was really really hard for me.

Talk to people! Just calm down and talk to people!
It seems quite obvious but, well, it isn't!

After a lot of talks with my manager, with some people from the office that went through the same shock ... I've decided to talk to my colleagues. After the talk, the situation has been improving quite a lot. And, for sure, what I felt during the first weeks is just a water over the bridge nowadays.

An important point to bring up here is the reason behind this whole situation. People (usually) won't speak their local language because they don't want you to understand the context or anything similar (although, is really hard to avoid that impression). Of course, it may happen, but in the most part of the cases it just happens because they don't even know the impact it may cause in their colleagues. In case it happens to you, at some point of your life, just let them know about the situation (in a polite way, of course) and I guess the outcome will be beneficial for both sides.

After the storm comes the calm
Now, after a few months working on SSSD (and together with FreeIPA) I finally feel comfortable enough to say that my decision to join this team was the really good.

During this whole changing period, some meetings happened inside Brno's office in order to find out what could be done to improve the "well-being" for foreigners here. Some points were raised, some changes already can be seen, which is quite good. Those changes have been pushed (mainly) by Jana Chvalkovska and her team. So, sincerely, I'm proud to see things moving to a good direction.

Also, as I mentioned before in the "Pain! A lot of pain!" section, a common cause for both problems that I had was my overweight. Well, I've been working on this as well. So far I've been able to lose 45+kg and there are 30kg more to go along this year that is coming. Let's see whether I'll be able to reach my goal ...

Am I happy in the end?
Yes. Happier than in the beginning of the year, for sure. Personal life has been going well. Work has been going well.

So, I guess there's nothing really serious that I could complain about ...

Any expectation for the next year?
As, at some point of my life, I've learned that the most effective way to avoid frustrations is to lower the expectations, let's say that outlive 2017 with less pain would be good enough. :-)
And if things keep going as they are right now there's a big chance it can easily happen.

Sunday, February 22, 2015

Willing to talk about Czech history? I'm willing to hear what you have to say!

Last Friday, when I was leaving Šelepka, I was approached by a small group of Czech people that, among other things, gave me a (though "non requested") quite nice lesson of Czech history. One of the guys, who seemed to be a History teacher, told us about Pod Kaštany, a concentration camp where over 600 people were executed or tortured to death ... and it was just in the place where we were. Although I have to admit that stopping people in the streets talking about concentration camps may not be the best approach to start a conversation, I was more than happy to learn a bit about what happened there.

An interesting thing is that they have a monument there that, even I'm passing by in front of it almost every day, I've never stopped and tried to understand what was written there. Quite sad, right? I do agree and that's the reason I'm trying to start a "project" (if I can call it in this way ...). Every Wednesday I'll try to go to Alterna to meet with Czech people willing to share histories about Brno (or Czech Republic in general), people that are willing to sit, drink a beer and talk about the history of their own Country. What can I offer for these people? Not much, actually. I can be a partner for beer, a good listener and, if agreed beforehand, I can share a bit of the Brazilian history (I'm kinda into social movements, women rights, gay rights, drugs' decriminalization ... and I can talk a bit about interesting stuff we had [or not] in all of these topics in the last centuries in Brazil). :-)

So, interested? Drop me a message and we can arrange something!

Saturday, November 29, 2014

Yesterday a Genius passed away

First thing, sorry for this completely non-technical post going to technical planets. If you don't want to ready about a completely emotional stuff, just stop reading now.

I'm a Brazilian that was born in the end of the 80's and grew up in the beginning of the 90's. Took me some time to have a telly at home, my parents were not so in favor of it. till the time they gave up following the religious rules ...
I remember the telly installed in a up corner of the room, it was a small one (14"). At that point I was experimenting a new world of four non-paid channels that we had access to. News, movies (from the 70's), culinary programs ... but one thing, in special, got my attention. It was a TV show about a "guy", with no name, with no family, living in a villa with: a poor widower dad and his smart daughter (the mom died during the birth), a middle-class widow mother and her spoiled son (the dad was a sailor that died in one of his travels), an old single woman (that loves the single dad), a teacher of the kids' school (in love with the single mom) and the owner of the villa, a rich and generous fat man that shows up every now and then to collect the rent, never paid by the widower dad. It was amazing, goddamn, it *is* amazing. A ~30 minutes episode (in Brazil we never had access to more than ~100 episodes), right after lunch, about this villa and the central character. The original name of the show is "El Chavo del 8". Episodes of "El Chavo del 8" where interleaved with episodes of "El Chapulín Colorado", a not strong, not smart, not fast, not powerful and not handsome hero, that always showed up to help anyone who asked for, usually in the most jumbled way, causing more confusion than actually helping. He was a perfect anti-hero. Goddamn, I cannot say how much I like these stupid TV shows. I remember my family sitting together and watching the Christmas special, the same one, for 10 years in a row. And it was better than the Christmas itself. Yesterday the genius who wrote these TV shows, and played the main character of both, passed way. Latino Americans with my age (+/- 10) are, for sure, sad today.
If you don't know this genius but are interested in: http://en.wikipedia.org/wiki/Chespirito
For those who feel the same pain, let's think that the man died, the legend will always be alive in our memories/hearts.

Buenas noches, maestro Bolaños!

Thursday, November 27, 2014

virt-viewer (using gtk3) for Windows

For the last few weeks I've been spending some time to have virt-viewer,
using gtk3, running on Windows. And it finally works!

Here is a not so brief description about the projects that I've hacked on (and the problems I've faced out during the process). if you're not interested in the process, just scroll down till the end of the email, where you will find the installer and a few instructions about what you need to generate the installer by yourself.

- msitools:
Apart from the broken build system on F21[0] and the wixl-heat generating the same ID for dirs used by other libraries/components[1] I have added .wxi files for the virt-viewer dependencies in order to have a build using gtk3. The dependencies are basically 3: 
- gtk3 itself[2]
- spice-gtk3[3]
- gtk-vnc2[4].
All the .wxi files were generated following the "How To" in the msitools webpage[5].
[0]: https://git.gnome.org/browse/msitools/commit/?id=c4c397438b797de4e68bb6d7fb891872db9e7bab
[1]: https://git.gnome.org/browse/msitools/commit/?id=41db587bae5a6c4cbc8f40a52bf2d6f25416653f
[2]: https://git.gnome.org/browse/msitools/commit/?id=c03676f95a348fd7da511355398f3a073400f819
[3]: https://git.gnome.org/browse/msitools/commit/?id=08cdc65a46f9eaaf49cd0acde0c3aaead341973d
[4]: https://git.gnome.org/browse/msitools/commit/?id=19c465b9f89583cdc905609abfd9cfb2c6beb26f
[5]: https://wiki.gnome.org/msitools/HowTo/CreateLibraryWxi

- virt-viewer:
Once that msitools has all the necessary bits included, it is time to hack a bit on virt-viewer and provide a way to generate the .msi file for virt-viewer using gtk3[6]. It is quite simple and basically a copy and paste from the file we already have been using for gtk2. just changing the dependencies and installation folder (just to avoid myself getting confused with the versions and so on).
[6]: https://fedorapeople.org/cgit/fidencio/public_git/virt-viewer.git/commit/?h=wip/msi-gtk3&id=4495b42a23ba5a795185bea0cadc540126b58751

After generate and install the .msi file on my Windows VM I crossed my fingers and ran remote-viewer!
Well, the results were not exactly the expected ones. A crash in libGdk-3.0.dll with no more info could be seen and also a lack of icons.

Okay, okay. First things first. Let me try to hunt this crash. Where can I start from? Hmmm. What about getting a debugger? :-)

- gdb:
Just got the binary from the mingw sourceforge webpage[7]
[7]: http://sourceforge.net/projects/mingw/files/MinGW/Extension/gdb/

Now I have the gdb, let me hunt the crash!
gdb.exe remote-viewer.exe ... and more problems! There were no debug-symbols in any of the packages I've been using ... and I need them!

- debugging:
 - approach 1 - -debug packages are your friends: well, not always. I've
installed the -debug packages and have added the new installed files
into the respective .wxi files. Then I've generated a new build and ... same crash, same poor info I had before (and nothing more). :-(
 - approach 2 - compile everything by yourself: this is what worked for me, but we have to agree that it's not exactly the most handy thing to do when you just want to debug something. Also, it inserted new bugs when launching the application, something about missing files that, for sure, were in the correct path. Anyways, it helped me to debug and I found that the Gdk crash could be avoided with a simple patch on spice-gtk[8].
[8]: http://cgit.freedesktop.org/spice/spice-gtk/commit/?id=b0703419bf15facc0527a81afbb3b07e67de909b

Yay! Now I have the virt-viewer working, but still no icons (apart from
the ones provided by virt-viewer itself).

For the icons problem, here is the basic solution:
- Build the .msi file using only the needed icons from the system installed adwaita-icon-theme. It works pretty well for virt-viewer, but if any other app tries to take advantage of the gtk3 on Windows, the app *must* have to do the same.

Yay, again! virt-viewer shows all the icons! Let's do some tests ... I've started by clicking in the menu options. The first one: "File -> Screenshot" ... and then crash!
Okay, this is an easy one. Basically I had to include the schemas.compiled[9] and schemas.dtd[10] into our gtk3 .wxi file, They are not being included because these files are not distributed with the .rpm, they are generated during the rpm's installation process.
[9]: https://fedorapeople.org/cgit/fidencio/public_git/msitools.git/commit/?h=wip/gtk3&id=e8990a125537ead4fad5d1adedc028cd72f5c932
[10]: https://fedorapeople.org/cgit/fidencio/public_git/msitools.git/commit/?h=wip/gtk3&id=97eefd9a3279a1766c81e54b0eb0e5660fece737

And now, finally, we are able to have a functional build of virt-viewer for windows using gtk3!

Do you want to try the build?

Do you want to build it by your self?
Okay, you'll need to:
- mingw-spice-gtk with my patch (build already pushed to f21):

- virt-viewer with my patches:

Compile and install msitools from git. Install mingw-spice-gtk and gtk-vnc2 packages. Then, inside the virt-viewer folder, do:
 $ mingw32-configure --with-spice-gtk --with-gtk=3.0
 $ make
 $ make -C data/ msi-gtk3

Next steps:
- generate the gtk2 or gtk3 build according to the configure options.
- add support for foreign menu for Windows.

Saturday, August 2, 2014

and GUADEC is over!

The conference was great, as usual, and quite more organized than I've expected (maybe I got a wrong impression due the issues that happened a few weeks before).
Kudos for Monsieur and Madame Franke for all the time spent on that. Special thanks for all volunteers as well!

I'm really happy that I had the opportunity to meet a few people there, that I've been working with but, till GUADEC, were just an IRC name. :-)

I want to thank Bastien Nocera, Felipe Borges, Rafael Martins, Victor Toso and Yuuma Sato (the dealers of Brazilian food), you guys made a couple really happy! \o/
I also want to thank Nimit for the Taj Mahal gift, man, it's awesome! And Parin/Nimit for testing Eliane's work and finding the (I hope so) last bugs to fix before push it upstream!

Next GUADEC is going to happen in Gothenburg, Sweden, the cradle of the Swedish Conspiracy and it will have at least a bit Carimbó! Maybe we will meet there again! :-)

Friday, July 11, 2014

Evolution-Data-Server: Backend-per-process!

I've been quite busy in the last weeks (months?) working in the latest big change in Evolution Data Server (EDS) and I'm glad to announce that EDS now has support to run each backend on its own process, what means that if one backend crashes, you can still have your whole factory (with another backends) running.
This change is enabled by default since commit f3f1e94f but the user also can have EDS behaving as it was before by simply passing  "--disable-backend-per-process" to the configure.

So, if you're running EDS from git master and have noticed any weird behavior with your application that could be a bug introduced by this task, please, file a bug in our bugzilla and put me cc'ed there. On the other hand, if you feel somehow happy with/thankful for this change, GUADEC is going to happen in 2 weeks, I'll be there and I do drink beer ;-)

A really big thanks to Milan Crha that reviewed the patches and was actively discussing this idea since the beginning of the prototyping part. Also thanks to Matthew Barnes and Tristan Van Berkom for sporadic help in #evolution channel, usually in the hours that only people living in America were awake :-)