Pierre Östlund

I think I’ve decided to go with an extension now. Been playing around with Thunderbird all day, trying to figure out how they work. So far I’ve been able to create the extension base (so that I can build and install a working extension) and I think I understand the basics well enough now. I finally know what XPCOM is and how it works too, only that is mind blowing

The biggest break through so far is that I’ve actually managed to figure out how the account manager works. I can now handle when accounts are added, changed or removed. Very basic, but it’s something.

I’m starting to question myself if the current approach I’m taking is really worth all the fuss. We will always have the problems with parsing Mork files and the backend itself is going to be incredibly complex all in all. Also got a response from David Bienvenu yesterday and he suggests writing an extension instead of taking the path I’m currently walking. This is the way both Spotlight (which David has been involved with) for Mac OS X and Google Desktop does it. It seems very rational and would make things a lot easier. We would lose the capability to index when Thunderbird isn’t running, but we would actually be able to “index” things when beagle isn’t running (the design would be similar to the IndexingService backend), if I decide to implement it this way.

I know that some people read this by now and I would like your input in the matter. Should I go on as I intended from the beginning or take the alternative road and write an extension (which would require a less complex backend) instead?

Also, David is putting some effort in adding support for loading individual mails in Thunderbird from the command line, which is one of the things I’m going to need in the future. We’ll see how this evolves, but it seems promising.

So, I’m not totally done with the Mork implementation yet - but there’s not much left. It’s just some basic things that I’m still unsure about (I’ve actually mailed a guy over at Mozilla about that) and some clean ups. But I still thought I would post some heap-buddy numbers to show the progress. Here’s the heap-buddy output from the old implementation:

Allocated Bytes: 55,6M
Allocated Objects: 943118
GCs: 17
Resizes: 12
Final heap size: 25,3M

Distinct Types: 114
Backtraces: 2269

As you can see, these numbers are very bad. Final heap size at 25,3MB isn’t that ideal (the file I’m parsing is 1,2MB in size). There’s also a lot of allocated objects, many more than necessary. This is not strange in any way since the old implementation used strings and regular expressions to work. The new implementation has ditched this concept totally and use only byte arrays. Now, here’s the current heap-buddy output:

Allocated Bytes: 4,8M
Allocated Objects: 150837
GCs: 15
Resizes: 11
Final heap size: 3,0M

Distinct Types: 92
Backtraces: 976

The numbers above are more rational than the first ones, but I’m still not 100% satisfied with them. They could be lower. But I can’t whine too much about it, it’s decent since the parser is fully implemented in managed code (=decreasing the speed and adding some memory overhead compared to native code and my code isn’t ideal either). And it’s also much much better than the previous implementation. Maybe I find a way to decrease the memory usage more later on, who knows. I also just want to point out that the numbers displayed here are preliminary and don’t reflect the finished implementation. There might be some variation, but I can’t know that yet.

I haven’t done any scientific speed measurings as of this time, but I do have a Beagle.Util.Stopwatch that gives me an idea of how fast the parser and database is. The old implementation currently take about 0,9 seconds to parse the same file I took the heap-buddy numbers from above. The new implementation needs only about 0,37 seconds, so it’s more than twice as fast. Which is nice.

So I’ve managed to reach the devilish point of this project: Mork. Used yesterday to brush up on it and created the various types that I will need to create my implementation. I’m currently hacking on some basic parsing methods and everything seems to be going pretty well so far. Since this implementation is stream based (reading byte by byte, the way it should be), adding debugging capabilities will be a breeze. One of the things I just added is line and position count so that debug message can give the exact position where a parsing error occurred. One of the major pros of this is that no one will have to send me their Mork files to me when debugging, just a few bytes around the trouble area so that I can locate the error. Much better from a privacy standpoint IMHO. Time to continue hacking!

I’ve just committed new code from the past week and sent my weekly report to the dashboard mailing list. It’s been a great week (I actually managed to complete my goals) and I’m very pleased. The upcoming week will probably be the most interesting and hardest week for me this summer: I will have to implement the Mork parser. Mork is this projects beast and I really hope to be able to complete it this week as I don’t want it haunting me. It would be nice to finish like a day or two early as well, leaving a few days for reviewing and testing before we enter next month (when I will begin to write the actual backend, woho!). I’ll leave it at that for know.

Update: The API is updated. Forgot to mention that.

Just wanted to give a small status report since last time (which would be last Friday). I’ve been working on the RangeList a lot, has taken some work but is mostly done and working. Might save some memory one day. The PreferenceParser (as mentioned in the API) is also almost finished. The remaining part is to fix a small bug in the FileReader (which is not yet in the API btw). I hope to be able to take Friday as my “update documentation and write tests/regression tests”-day so that I can post them here, in case anybody wants to help out by testing my accomplishments.

I also want to point out that the API is quite out of date at the moment and will be updated sometime this Friday.

Finally - I’ve set up a new Subversion branch and uploaded some code. It’s all stubs and can’t be used for anything yet. So… filling all the blank spots and writing some code will be the next step. All planning and documenting is pretty boring when one do it to much, so I need to get some coding done to increase my motivation a bit.

I’ve added a lot of things to my new branch, so just check it out if you are interested. [1] There’s also an “API” page with some reference material as well. [2]

If you want to do an anonymous check out, you only have to type:

svn co http://svn.gnome.org/svn/beagle/branches/beagle-tbird-soc07

…to get a local copy. Just compile and install it as usual.

[1] Browse the code

[2] API reference

It’s been final for a while now, but it seems like Richard Stallman is coming to Linköping to give a guest speech this Thursday. The topic of the evening will be “Copyright vs Community in the Age of Computer Networks” and will most likely be very interesting. This lecture is a bi product of Stallman’s speech in Gothenburg this Wednesday[1] and is arranged by Digifri[2] (a student association here at the university).

What’s else on the agenda… a whole lot. This semester is almost over and there’s a lot to do right now so I haven’t had much time to work on beagle at all. Hopefully I’ll get some time next week, otherwise I’ll make some.

[1] http://rms2007.se/in-english/

[2] http://www.digifri.org/en

It has been a busy week and school is currently taking a lot of time. That’s why I haven’t had much time to work on my project the past week (just on some very small things), but hopefully it’ll get better soonish. Just sent my papers to Google though, so hopefully I’ll get all that sorted out. That’s about it for now.

This year’s Summer of Code projects were recently announced and my application was accepted! The topic of my application was Thunderbird backend rewrite, which is pretty self-explaining IMHO (press the link to read a very short abstract). So… I will apparently spend my summer working on an Open source project that I really enjoy - awesome! I have already started with the planning and will hopefully publish a more detailed description of my project soon. Check back later.

As this is my first blog entry, a short introduction to myself would probably be in order. My name is Pierre Östlund and I’m a 19-year-old first year computer science student at Linköping University, Sweden. Been using open source software a couple of years now and decided that I wanted to give something back to the community. Beagle was the project that caught my eyes and was the project I began to write code for about a year ago. The Thunderbird (including the contact viewer) and Opera backend are mainly my accomplishments so far, but I hope to contribute much more in the future. When I’m not in front of my computer or in school, I usually spend some time with my friends and/or play some Wii.

I have updated the beagle wiki with some information about my project. Check it out if you want to know more about my SoC project.

http://beagle-project.org/Summer_Of_Code_2007