Libre Software People's Front

don't confuse it with People's Front of Open Source

How to get quantitative data from the Android source code (I)

with one comment

One of my targets for 2011 is to make as easy as possible the process of obtaining quantitative data from open source projects. We have developed several tools with that purpose but they still need a lot of love to be really user-friendly and stable. In the following two posts I’ll show you how to get basic data from FLOSS projects using the source code repository, in this example we will study the code provided by Android using cvsanaly to get data from the repositories and R to create a couple of charts.

The Google developers created a tool called repo to deal with the different git repos that they are using in Android. I don’t like to install tools that I won’t use so I’ll bypass it with a couple of bash commands.

The repo command uses the git:// as starting point, so after cloning this repository you’ll see that it contains a XML file called default.xml with the following content:

  <project path="system/bluetooth" name="platform/system/bluetooth" />
  <project path="system/core" name="platform/system/core" />
  <project path="system/extras" name="platform/system/extras" />
  <project path="system/netd" name="platform/system/netd" />
  <project path="system/vold" name="platform/system/vold" />
  <project path="system/wlan/ti" name="platform/system/wlan/ti" />

The XML code above only shows some of the 159 references to git repositories. Without the repo command created by Google, the developers should have to download them one by one or using a script. We will use awk and a simple bash script to extract them form the XML file and download them in one go.

$ list=`cat default.xml |awk -F '"' '{print $4}'|grep -v '^$'|grep -v "UTF-8"|grep -v "Makefile$"`
$ for i in $list
j=`echo $i|sed 's:/:_:g'`
echo git clone git://$i $j >>

Now, just edit the file and add the following lines at the beginning and we have a script to download the Android’s repositories. Don’t forget to give it execution permission.

echo "getting android repos"

Easy, isn’t it?. The next step is to execute the script to download the 159 git repositories and in the meanwhile install cvsanaly which has to be installed from sources, but do not panic it is straightforward:

At this point you are ready to start playing with the raw data extracted from all the git repositories in a single relational database. Stay tuned, the second chapter is coming soon.

UPDATE: the new release of Android 2.3 which has been published a couples of days ago uses 167 git repositories

Read the second part


Written by sanacl

December 17, 2010 at 8:52 am

One Response

Subscribe to comments with RSS.

  1. […] have a look at the previous post if you didn’t […]

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Google+ photo

You are commenting using your Google+ account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )


Connecting to %s

%d bloggers like this: