Friday, July 11, 2014

Evolution-Data-Server: Backend-per-process!

I've been quite busy in the last weeks (months?) working in the latest big change in Evolution Data Server (EDS) and I'm glad to announce that EDS now has support to run each backend on its own process, what means that if one backend crashes, you can still have your whole factory (with another backends) running.
This change is enabled by default since commit f3f1e94f but the user also can have EDS behaving as it was before by simply passing  "--disable-backend-per-process" to the configure.

So, if you're running EDS from git master and have noticed any weird behavior with your application that could be a bug introduced by this task, please, file a bug in our bugzilla and put me cc'ed there. On the other hand, if you feel somehow happy with/thankful for this change, GUADEC is going to happen in 2 weeks, I'll be there and I do drink beer ;-)

A really big thanks to Milan Crha that reviewed the patches and was actively discussing this idea since the beginning of the prototyping part. Also thanks to Matthew Barnes and Tristan Van Berkom for sporadic help in #evolution channel, usually in the hours that only people living in America were awake :-)

Thursday, February 27, 2014

Evolution ideas for GSoC 2014!

I'm happy to announce the Evolution team has a few ideas of projects for this GSoC round!

Complete introspection support for EDS:
This is for evolution-data-server, but only marginally, because the most work might be done in libical, to provide a GObject-based interface for their icalcomponent/icaltimezone/... all structures, basically.
As a starter, the ECalComponent (the wrapper around icalcomponent) can be extended and made introspactable too (it is a GObject descendant already, but some structures are raw structures). The biggest problem is that the ECalComponent can hold only one icalcomponent, not a set of icalcomponents, which is required by ECalClient API, to be able to fetch events with timezones or a recurring event with detached instances. The student may see what is required by the ECalClient API and then propose "the easiest way".

Standalone app for editing server-side Sieve filters:
This doesn't touch evolution's code directly, as it's a prove of concept and a test application to allow editing of Sieve filters directly in evolution. Having it in mind, I'd like to see students with previous Sieve experience and able to discuss the project in #evolution and that end up with a prototype about what will be implemented 'til the end of the applications period.

Notmuch as indexer and search language:
Ideally, a student may create a new CamelIndex descendant, which will talk to Notmuch in the background, and will be usable with CamelFolderSummary (its camel_folder_summary_set_index()). Then the student will touch camel-folder-search.c, because it also uses CamelIndex for body-searches (that's basically the only usage of CamelIndex, for body-searches, but I know the notmuch knows much more).
A similar changes may come into camel-filter-driver.c, together with an addition of "notmuch-code" expression, which will require notmuch for providing output. Then, in evolution, the mail/vfoldertypes.xml,
mail/foltertypes.xml and mail/searchtypes.xml may be extended to also support "notmuch-code" expressions.
As the notmuch project is too much for a SoC period, we would like to have it done only for local Maildir accounts :-)

Implement a simple PKCS#11 module using evolution's addressbook as a backend:
Nowadays we have an address book, and it contains details of X509 certificates for people. But that information is basically in the wrong place. When we come to do S/MIME signing or encryptionm we need the NSS library to be able to find the certificates by the normal lookup process that it uses. In the certificate database(s). So the idea is that the student provides a PKCS#11 module which appears to contain all the keys that are found in the address. When it's asked to search for a certain key, it goes and looks in the addressbook for matching user with an E_CONTACT_X509_CERT field in their addressbook entry and returns that cert, if found.

A nice way to start working on this idea was described on #evolution by David Woodhouse:
<dwmw2> I'd start simple; create a trivial PKCS#11 module with a *hardcoded* cert in it. Just one
<dwmw2> so you can send email to that user and it can find that cert in your module
<dwmw2> that'll get you putting together the basic guts of the PKCS#11 module with its search and get certs functions.
<dwmw2> and wiring it into the right place in Evolution to *use* it
<dwmw2> then after that, we make it a proper addressbook client and make it do the right search in the addressbook for *real* data.


If something is not clear and/or you want to discuss the ideas, please visit the #evolution channel or send us an email through evolution-hackers mailing list (https://mail.gnome.org/mailman/listinfo/evolution-hackers) and we will be glad to help with anything that is necessary.

Have fun!

Wednesday, January 15, 2014

Evolution EWS: TimeZones operations

Long story short:
For those who are using evolution-ews from git master, calendar is  back to the game.

Long story:
With the changes I've made to add support for streaming notifications I've also introduced a big bug.
Calendar was not working anymore for Exchange servers newer than 2010.

Why?
Because newer versions of EWS do handle an event time in a completely different way than older versions and I had to add support to a new operation that was not supported from our side so far.
.
Why? I thought we were already using the proper server version to talk to EWS servers (damned, I would love have it blog posted, unfortunately I don't have).
Hmmm. Yes. Or better, not exactly. When I've made the changes to speak to the server using the proper server version, instead of use the oldest one, I didn't touch in the calendar part, because it would take a not small time to implement this operation and so on. With the streaming notifications code added, I had to do this changes (once it works only for EWS servers 2010_SP1 or newer). And ... Boom! ... I broke the calendar backend.

And now it is working again, right?
Yes, it is! :-)
The code is still not reviewed but can be used/seen from/on this branch.

By the way, tests were written and the calendar backend code was refactored in favor to reuse backend's code in our tests, and it took me considerably more time than adding the new operation itself. :-p

Sunday, January 5, 2014

Google Gym!

During my vacations, I traveled to cities I didn't know, having to use Google Maps and trusting in its integration with public transportation, what is the motivation of this post.
That Google is always looking on our searches and using it to offer us "personalized" advertisements is not something new, but that Google does care about your health care is something , at least, unexpected. If you, reader, do not know me in person, let me tell you, I'm fat. I'm not a bit overweight, that guy that ate a bit more than he should. I'm really fat ;-) And when I'm travelling my internet usage can be summarized in basically 2 things: look for a place to eat and get the directions to go to this place (of course, walking less possible).
But, as I said before. Google does care about my health care. More than I do, apparently. Looking for the "less walking" and "best route" options from one place to another, with a 100% of accuracy, Google provided me the "more walking" route possible. In the first 2, 3 times I need to admit, I was completely pissed off. After that, I started to realized they were testing their new product and I am one the of luckiest guys in the earth to have access to it first: Google Gym!
And what the heck is Google Gym? Simple, based on your searches Google will calculate how many calories you're eating in each meal and based on your pictures Google will decide how many calories you should eat in each meal. Do you eat more than you should? Okay, Google makes you walk for 30 minutes next time you're going to a restaurant! Com'on, it's better than a personal trainer!

Google, please, if I am not in the test group and everything said above is just part of my sick mind, please, consider start to implement it. People would love it! (And, please, remember this blog post and send me a gift :-)). Or, at least, fix the map routes because it's getting worst since last maps update :-p

Tuesday, November 12, 2013

Evolution EWS: Notification Operations (only for Exchange Servers 2010 SP1 or newer)

The news:
Notification Operations are now pushed to evolution-ews git and will be available on 3.12 release.

Small demos:
You can have a good idea about what I'm talking about taking a look on these videos, where you can see:
- On the right side of my screen you can see the Outlook Web Access from where I'm playing with my account.
- On the left side, you can see my EWS account on Evolution, receiving notifications and updating the content (to reflect with the changes made on the right side) almost in real time. :-)

How can I enable it?
In your EWS account's preferences, go to "Receiving Options" tab and enable "Listen for server change notifications".
If you want to listen for notification in all folders, also enable "Check for new messages in all folders", otherwise only your Inbox will be updated.

Limitations (Please, read it, this is the most important part!):
- It only works for Exchange Server 2010 SP1 or newer.
- However our API has full support to subscribe any folder you want to listen the notifications for, for now it is only Inbox or Everything. I would like to avoid big UI changes for this cycle.

Microsoft documentation about this feature:
http://msdn.microsoft.com/en-us/library/hh312849(v=exchg.140).aspx


Thursday, October 10, 2013

Evolution EWS: Testing EEwsConnection's API

Howdy!

tl;dr:
EEwsConnection's API has a new test framework.
We only cover folder operations for now (Create, Delete, Move)
Please, help us! :-)

verbose:
I'm glad to announce that we are adding low level tests to cover EEwsConnection's API. It is and will be a continuous work until we have the (almost) full coverage done.

How are we doing this?
It's simple. We are mocking the EWS server, collecting the traces (what means, we are testing against a real server) and then we have the possibility to test our changes offline (against the traces collected). For this, we are using a library written by Philip Withnall, announced during GUADEC, named uhttpmock.

How is the coverage for now?
As I wrote it as a proof-of-conception, for now we are barely covering folder operations (Create, Copy, Move), but the idea is add support while we are touching in another parts to fix bugs/add new features.

Which EWS versions are being covered?
For now, Exchange2007_SP1 and Exchange2010_SP2. I only have access to these servers (actually, we also have access to an Exchange2013 server, but we only will have support to send requests with this specific version when we have a more complete coverage and then track the breakages before commit this new support. Ah, so EWS doesn't have support for Exchange2013? False! We do, but we do the requests as an Exchange2010_SP2 server). So, if you have access to servers with different versions, please, feel free to send us patches to help us to improve the quality of our tests.

How could I help?
If you have access to an EWS server, please, take a look in our tests (they are really simple to add) and start to create new tests in the area of your interest. We would be really happy with some help. We need some help :-)

That's all folks!

Monday, August 26, 2013

Evolution EWS: Planning the 3.12

After a release cycle more focused on bug fixes, now is time to sit and plan what I will do want to do for 3.12.

tl;dr version:
  • Notifications operations
  • EmptyFolder operation
  • (improve) FindItem operation
  • GetRooms and GetRoomLists operations
  • Time zone operation
  • Sharing operations
  • Unit testing
To check what those operations mean, take a look into: https://wiki.gnome.org/Apps/Evolution/EWS_OperationsFeaturesParityMatrix

Okay, now in a (kind of) verbose mode:
  • Notifications operations: I have this feature half-implemented, still with some bugs related to the subscription timeout, mainly when the connection is somehow cancelled (or evolution is turned to work in offline mode, or the server is down). It involves a big work on addressbook, calendar and mail (camel) parts to apply all received events about changes; a dozen of APIs in the server communication part to subscribe/unsubscribe to listen those notifications, parse the received events and provide them to all backends that want to listen for notifications; some code refactoring in the *_start_sync backends methods; and, finally, some simple modifications in the UI, to enable/disable this feature.
  • EmptyFolder operation: We already support a folder deletion in Evolution EWS, so this is more like a good improvement in the way to do the deletion than a super-duper feature.
  • (improve) FindItem operation: We have a first implementation of this, what allows us to use the server to do some searches for us and then we only handle the result. Hope we can, at least, fix this bug improving this operation, let's see.
  • GetRooms and GetRoomLists operations: Yeah, we are missing these ones. Basically we can get the rooms that are available within the Exchange organization
  • Time zone operation: We have our own way to handle the time zones in Evolution, but it is useful mainly to check if we are not trying to set an invalid time zone for the EWS server. Ah, taking advantage that I'll be working with the time zone operation, I would like to do a small code refactoring the calendar backend to use our server calls based on the server version, that is already done by the another backends but get stucked to fix it for calendar, since the time zone operations are quite different from one version to another (thanks little Bill).
  • Sharing operations: We already have an implementation for this. On the one hand it is not using the operations introduced by EWS 2010. On the other hand, it works with an older version (EWS 2007), what means that we doesn't have Discovery neither Accept for sharing invitation working. What I want to have here are the "new" operations working properly with newer and older severs without mess with the whole EWS 2007 support.
  • Unit testing: This one deserves another blog post, coming soon. :-)
And that is it for now!