So I've spent quite some time in the last couple of months (whenever I had a chance, actually), to redesign how the core of Planeta Linux works, and I'll explain all the changes I've made to make it a much better solution that fully fits our current needs. If you are reading this with a feed reader, star this item, share the item, retweet the post! Let everyone know we are doing this for the Linux Latin American community!

If you're naive enough, you'll think about Planeta Linux as a simple Planet aggregation instance with a different set of people collections, which we used to call, ironically, instances, which aren't anything else but countries. If you don't know how we've been doing Planeta Linux for almost five years now, you'll think it's all just config.ini being fed to the planetplanet binary. Well, it has been like that, yes, but having a big set of planets (more than ten now) makes it a bit of a hassle to do any changes or try to do anything else but add, edit and remove people. The way we were handling it with Apache was already documented in the past.

Some of the multiple issues Planeta Linux has had in the past could be summarized as:

• Inability to have a unified templates set.
• Repeated feeds of the same person on multiple countries.
• Inability to know when a feed started to properly return valid items.
• Error-prone parsing.
• Difficulty to test on local environments.
• Decentralized subdomains with difficulty on sharing content or awkward navigation between contries.

And the list could go on. I realized this was a huge pain in the ass when I had the intention to centralize the subdomains into a single one: Instead of using the model $country.planetalinux.org, everything would have to come out from just planetalinux.org. Changing to that on our previous scheme was simply an hemorrhoid. So, I changed it all and the new model is much better and robust, even though you probably don't see a single difference on the current Planeta Linux output.

I started by getting rid of the horrible config.ini editing. Whenever you want to add an author, you simple drop a YAML file into the authors directory. It doesn't matter the location of the file as long as it's inside the authors directory. When I started importing all of the subscribers, I ended up with more than 620+ author files so I made a simple separation of files a-la CPAN: authors/d/da/david_moreno.yml. But at the end, it doesn't really matter, a) what your file is named, and b) where you put it, they will all get parsed and interpreted. Now, inside that YAML file there's a bunch of goodies. Firstly, you have the feed URL, the name of the person, his or her email, an array of countries where the feed should be aggregated into and other stuff such as the path for the hackergotchi, a twitter parameter for the Twitter feed, an enabled switch, etc. The good thing about it is that you can have all the parameters you want so we can build on top of that later (maybe identica profiles, LastFM usernames, whatever).

One of my future intentions is to have also an array of feeds on each author file which would empower us to have the same info for several feeds. With the new model, this should be much easier to implement anyway.

Anyway, basically, when you want to add or edit a feed, you just edit or drop the file on the authors directory. KISS.

Second big change, we switched to Planet Venus, as opposed to just Planet for the aggregation engine and you no longer need to have it installed since it's included now on the entire Planeta Linux tree under the venus directory. Venus implements a whole new world of features and good stuff on top of Planet, it was just a no-brainer to switch to it.

However, Venus still requires config.ini for each of its instances, so this is where the whole build process comes to play. I implemented a tool, script/build that does exactly that, it gathers all the info from authors and builds each of the countries. To create a new country, you just basically only have to add it to config/countries.list. The build script will use all of them as tasks and execute them at will.

Now, how the build tool works? As said, it gathers the data from authors and generates a config.ini on the fly. There's a config.ini template file on the template directory, as well as an index.html template (and rss.xml), and for the build script, it's just another Template Toolkit file so you can do all sorts of awesome and crazy shit with it. So it's all generated dynamically, with the information from authors, information from the config directory and it dumps it all on the www directory. Because of that, you can generate your own working copy of Planeta Linux or implement others or just play with it!

So basically, the whole process is like this for each of the countries:

The good thing about it is that there's no interaction for the administrator to deal with the config.ini either when adding an instance since I integrated a tool that adds a new subscriber to the authors tree automatically:

~/axiombox/planetalinux :master $ ./script/planetalinux.pl help add
planetalinux.pl add [-p] [long options...]

Adds a new author to Planeta Linux. The name, email, feed URL and
country where to place the author are mandatory. If the hackergotchi
image path is provided, the script will check the size for the image
and resize it if needed (ImageMagick needed). Any other flags and
values passed to this command will be appended on the resulting
YAML file.

Examples:

./script/planetalinux.pl add \
--feed http://example.com/feed \
--name "Tía Chonita" \
--email tia@chonita.com \
--countries ve \
--hackergotchi ~/images/chonita.jpg \
--twitter @chonita

./script/planetalinux.pl add \
--feed http://blog.wordpress.com/feed/atom \
--name "Isela Crelló" \
--email yeah@yeah.com.mx \
--countries mx,sv,gt \

./script/planetalinux.pl add \
--feed http://cofradia.sucks/feed \
--portal \
...etc

--feed feed URL -- mandatory
--name name of author of feed -- mandatory
--email email of author of feed -- mandatory
--countries country(ies) of author -- mandatory
--hackergotchi path to the hackergotchi image -- optional
-p --portal portal site flag -- optional
--twitter twitter feed of author -- optional

~/axiombox/planetalinux :master $

Cool things about this new "adder" script:

• It ensures that the author has an stored feed, name, email and country, at the very least.
• It checks whether the URL is a valid feed URL against the W3C feed validator.
• It checks that the email is valid.
• It checks that the countries specified are supported by the system.
• It takes a file path for the hackergotchi and using ImageMagick, it resizes it, converts it into the proper image type and places it under the appropiate image heads path.
• It creates the YAML and places under an appropriate location under the authors directory.
• It's awesome

Tutorial

Now, one of the good things about all of this is that you can create your own Planeta Linux running right there on your machine, given that well, it's just a nice big glue involving Perl for the processing, Python for the parsing and aggregation, the cache is stored on the tree as well, etc.

To start, you'll have to clone the repository if you haven't done it yet:

$ git clone git://github.com/axiombox/planetalinux.git

Change the directory to the repository and just run the dependencies installer for all of the modules used:

$ sudo perl installdeps.pl

Note that you'll have to be running perl 5.10 (it's been stable for almost two years now, dude! Upgrade!). If you already have most of the modules installed on your system, this shouldn't take that long. If you have nearly no Perl modules installed, it will probably take a little while. It might even need your intervention for some basic CPAN configuration. Once it's done, you should see something like this, at the end of the output:

.. testing loading modules...
- App::Cmd
- App::PPBuild
- Config::IniFiles
- Data::Validate::Email
- DateTime
- File::MimeInfo::Simple
- File::Path
- Modern::Perl
- Net::Domain::ES::ccTLD
- Template
- WebService::Validator::Feed::W3C
- YAML::Syck

.. done. enjoy!

.. please make sure you have PerlMagick installed.
.. the recommended way is: `port install p5-perlmagick` in Mac,
.. ..or 'aptitude install perlmagick' in Debian/Ubuntu.

Now, for the ImageMagick Perl bindings (for the "adder" funcionality) you'll need to install it depending on your operating system. If you are running some flavor of Debian (or Ubuntu), you just have to install the perlmagick package. On MacOS, I recommend installing the ImageMagick MacPorts port with the "+perl" flag. Depending on your configuration, you might need to install the Image::Magick module either from the CPAN shell or downloading it from web. Once all of that is done, you can just fire up the building system:

$ ./script/build all

You can just fire up a single country or other tasks by seeing which ones are available:

$ ./script/build --tasks

At this point you can go to your browser and navigate to the www directory on the repository where all the output has been dumped.

If you have any issues running your own local copy of Planeta Linux, please put a comment on this post, I'll glad you help you solve it and you'll be helping us making Planeta Linux much better than ever.

This next Tuesday October 20th, the Perl Seminar returns to New York City! This is a great effort by Jim Keenan, always trying to push the meetings and encouraging people on attending and sharing. There's usually one meeting per month and it's very well located, in midtown Manhattan, on a suite in the Ramada New Yorker hotel building facilitated by the NYPC User Group.

The first meeting will be about Git, GitHub and how you can use it on your Perl code and projects. It'll be very interesting for newcomers! More details on the original post, and make sure to subscribe to the discussion group.

I attended YAPC|10 during this past week. The overall experience was quite particular, with ups and downs, but after all I made it back in one piece.

I rode a Greyhound bus on my crusade of saving a few bucks. I always had issues with buses, when I was a child I used to go from Celaya to Mexico City (and back) almost every weekend for a couple of years and I developed a phobia against the whole bus experience. Even though, getting to Pittsburgh from New York City was cheaper riding a bus than the other alternatives anyway, flying or train. Anyways, it was an awful experience that prevented me from being complete recovered after each travel leg.

Staying at the CMU dorms was also a nice and pleasant experience, feeling for three days like I was a kid on a nice American college. I might want to do that some time in the future pursuing a Master's degree (some day).

At YAPC, I had the nice chance to get to know a lot of people I'd only known by mailing lists and CPAN modules authors that I use on a daily basis. I also had attended YAPC 08 last year in Chicago, so I already knew a small group of mongers.

I delivered my talk about Perl and social networks on Tuesday. I gave it an small twist by moving it into a more motivational, business perspective for hackers, in the last minute. I was overall satisfied with it, and I also think some things were not well done and had to be changed. Fortunately, quite some people came to me and gave good, positive, constructive feedback about it, that I was very thankful to receive, and unfortunately some people also threw destructive criticism that only makes you think how douche-baggy people in the community you can be. In any way, I was happy and satisfied, some observations:

• I have to take out a lot of personal stuff I poured into it because people don't care (and it's not their fault, it's mine).
• Focusing more on actual code. I can try to focus on motivating people to adventure into social networks application development using Perl, but the YAPC audience is more into actual development and coding techniques than money-making ideas.
• I still have to practice my speech. I took the time to write my lines scripts for each one of the slides, but that makes the interaction with the people a bit more difficult to establish.

And some interesting points around YAPC in general:

• In the auction, event that is done every year helping TPF raise some funds and that a similar approach would be great to have in conferences like DebConf, the highest bid for a lunch with Larry Wall reached one thousand dollars. I was a bit surprised.
• They took us in a tour at Heinz Field, home of the Pittsburgh Steelers, it was awesome, although it was already a bit late and couldn't take good pictures of the field. Pictures are on this Flickr set.
• I had some time at the end of my stay in Pittsburgh so I took the opportunity to go to PNC Park and watch the Pittsburgh Pirates kick one of their greatest rivals (as I learned there, because of the proximity), the Cleveland Indians. Pictures are on this set.
• Some people outside the venue also joined me on the conference as I setup an improvised ustream transmission. Unfortunately, I never got to press the "Start recording" button, so no video of it. Bummer. However, God willing, I will record my next delivery of the talk on YAPC::EU in Lisbon, in early August.

I've placed the slides on a PDF here. You can also clone the entire repository containing the source Keynote file.

Make sure you add the just launched git ready blog to your feed reader. Made by qrush. Audience applauds!

Following the Git tips being thrown on Planet Debian, here's a couple:

1. Changing into a directory that contains a repo and shows you on PS1 what branch you are standing on:

On .bashrc I have:

GITPS1='$(__git_ps1 ":%s ")'
export PS1="${GREEN}\w${RS} ${YELLOW}${GITPS1}${RS}\\$ "

2. An alias I like to use on repos that are personal for quick tracking:

[alias]
  ...
  log1 = log --pretty=oneline --abbrev-commit

I hope you like them.