Friday, November 14. 2008
"If we have 2 or 3 good services at launch I will be happy"
With Amarok 2.0.0 rc1 right around the corner, now is a good time to reflect on where Amarok 2 comes from and where it is going. So I felt like writing a bit about the journey of the idea of "services" in Amarok 2, as that has been my main focus, even though I have managed to get my hands dirty all over the place it seems!
Just over 2 years ago, Amarok 1.4.4 was released with a cool new feature, which also happens to be my first contribution to Amarok, the integrated Magnatune.com store. ( Here is a cool page that Magnatune did to document some of the responses ). The overall response to this was quite good, and Magnatune started selling quite a few albums through Amarok, and eventually ended up hiring me, and I still work for them.
Something else started happening as well. People saw the integrated Magnatune store and started asking if there was any chance that their favorite store could get a similar integration. Most of the Amarokers agreed that this could be cool, but there were several obstacles. For one, the way the original Magnatune store required a huge amount of custom code to do simple stuff like adding tracks to the playlist ( and as many will likely know ) the metadata representation of these are not perfect. Also, The Magnatune store had its very own tab on the left side of Amarok, and it was clear that we could not just add an arbitrary number of these as we started to add more stores. Finally, the Magnatune store in the 1.4.x series of Amarok is tied very closely to the rest of Amarok, meaning that it cannot easily be removed, and that people are more or less forced to load part of this code, whether they use the store or not. Luckily for me, after a time, something big happened in Amarok-land, the 1.4.x series was put on maintenance mode and the work on Amarok 2 was started. Since I was only responsible for porting over the Magnatune store and had almost no other code in Amarok, I decided that this would be a good time to try to tackle some of the issues mentioned above, and prepare Amarok for further stores or other services to be integrated.
To make a long story short, we now, after a year and a half of work, have a framework in place that allows services to be implemented as plugins and loaded/unloaded on demand, a service browser to show them in and overall much better integration into Amarok overall, basically solving all the issues that needed to be solved before we could add more services.
My original plan was to port at least the Magnatune store to this new framework, and as the title of this post shows, when I started this work, I would be very happy to have 2 or 3 working services to show up when 2.0.0 was launched. As the image on the left shows, this is not quite what happened.
This image shows the services that are currently available, either included with Amarok 2 itself, or via download from kde-apps.org ( easily installable from within Amarok 2 using the "Get Hot New Stuff" system ). Some of these services are coded using the C++ framework, and some are scripts that run on top of the "Scriptable Service" framework, which is itself an extension of the underlying service framework. I have done a number of them myself, but more and more services are added or maintained by others. There are 13 of them. This is way more than I had ever hoped we would have available anytime soon, and really shows off the power of the framework well. Especially the scripted service framework, that lets people relatively easily add content from an online source ( although in a somewhat limited way compared to a full C++ plugin ) has received a lot of interest lately, and these scripts seem to be pouring in at the moment.
Looking at the picture of all these services, one does start to appreciate how useful it is to be able to only load the services that are interesting to you, and not having to have them all in the list all the time!
So what will the future bring? For starters, I have realized that I might need to extend the API used by service scripts a bit, since these seem to really be taking off in a big way, and requests for new features are already coming in ( and some have already been implemented ). Beyond that, I know of quite a few services that are being worked on, or are in the planing phase, both scripts and very advanced full plugins, so as with the rest of Amarok 2, this is not the end result, it is merely the beginning, and cool things will happen over the next many years, as we fully realize the potential of the new codebase!
Just over 2 years ago, Amarok 1.4.4 was released with a cool new feature, which also happens to be my first contribution to Amarok, the integrated Magnatune.com store. ( Here is a cool page that Magnatune did to document some of the responses ). The overall response to this was quite good, and Magnatune started selling quite a few albums through Amarok, and eventually ended up hiring me, and I still work for them. Something else started happening as well. People saw the integrated Magnatune store and started asking if there was any chance that their favorite store could get a similar integration. Most of the Amarokers agreed that this could be cool, but there were several obstacles. For one, the way the original Magnatune store required a huge amount of custom code to do simple stuff like adding tracks to the playlist ( and as many will likely know ) the metadata representation of these are not perfect. Also, The Magnatune store had its very own tab on the left side of Amarok, and it was clear that we could not just add an arbitrary number of these as we started to add more stores. Finally, the Magnatune store in the 1.4.x series of Amarok is tied very closely to the rest of Amarok, meaning that it cannot easily be removed, and that people are more or less forced to load part of this code, whether they use the store or not. Luckily for me, after a time, something big happened in Amarok-land, the 1.4.x series was put on maintenance mode and the work on Amarok 2 was started. Since I was only responsible for porting over the Magnatune store and had almost no other code in Amarok, I decided that this would be a good time to try to tackle some of the issues mentioned above, and prepare Amarok for further stores or other services to be integrated.
To make a long story short, we now, after a year and a half of work, have a framework in place that allows services to be implemented as plugins and loaded/unloaded on demand, a service browser to show them in and overall much better integration into Amarok overall, basically solving all the issues that needed to be solved before we could add more services.
My original plan was to port at least the Magnatune store to this new framework, and as the title of this post shows, when I started this work, I would be very happy to have 2 or 3 working services to show up when 2.0.0 was launched. As the image on the left shows, this is not quite what happened.
This image shows the services that are currently available, either included with Amarok 2 itself, or via download from kde-apps.org ( easily installable from within Amarok 2 using the "Get Hot New Stuff" system ). Some of these services are coded using the C++ framework, and some are scripts that run on top of the "Scriptable Service" framework, which is itself an extension of the underlying service framework. I have done a number of them myself, but more and more services are added or maintained by others. There are 13 of them. This is way more than I had ever hoped we would have available anytime soon, and really shows off the power of the framework well. Especially the scripted service framework, that lets people relatively easily add content from an online source ( although in a somewhat limited way compared to a full C++ plugin ) has received a lot of interest lately, and these scripts seem to be pouring in at the moment.
Looking at the picture of all these services, one does start to appreciate how useful it is to be able to only load the services that are interesting to you, and not having to have them all in the list all the time!
So what will the future bring? For starters, I have realized that I might need to extend the API used by service scripts a bit, since these seem to really be taking off in a big way, and requests for new features are already coming in ( and some have already been implemented ). Beyond that, I know of quite a few services that are being worked on, or are in the planing phase, both scripts and very advanced full plugins, so as with the rest of Amarok 2, this is not the end result, it is merely the beginning, and cool things will happen over the next many years, as we fully realize the potential of the new codebase!
Trackbacks
Trackback specific URI for this entry
No Trackbacks
Comments
Display comments as
(Linear | Threaded)
That so many scripts already exist and that 3rd party folk are jumping on board is a great testament to you plugin framework.
Could you perhaps write a blog entry or even techbase tutorial on what your design considerations were and the technical aspects of implementing such a framework? The way that service extensible architectures such as this and the one in plasma really open up the environment for all kinds of creativity and I'm sure there are many other areas that could benefit!
Could you perhaps write a blog entry or even techbase tutorial on what your design considerations were and the technical aspects of implementing such a framework? The way that service extensible architectures such as this and the one in plasma really open up the environment for all kinds of creativity and I'm sure there are many other areas that could benefit!
This might sound extremely odd, but my advice to anyone who wants to build a framework like this is: Don't!!
This is not me trying to be funny though. The point is that I could never in a million years have designed the Service Framework, and if you look at the history of the services and the framework, you will realize that I really didn't. What I did do was build one service, the Magnatune store, and then extrapolate the best parts of that ( the parts that really proved useful, and not the parts that I thought in advance would be useful ). I then used these parts to build the next service, Jamendo.com, and then extracted some more bits.
Doing development this way does cost you quite a few rewrites of stuff you have already done, but the benefit is that you don't start out by trying to look far into the future, trying to anticipate every possible feature that services some time in the future might require. This will in my experience only lead to you implementing tons of stuff that you do not really need, and missing bunch of features that you really should have been doing instead. By letting a framework evolve rather than designing it up front, it will likely become less complex and much better adapted to the stuff that it actually ends up being used for.
I should stop now. You sort of hit on of my push buttons I am actually a certified scrum master and have worked quite extensively with lean and agile methodologies in one of my former jobs, and not designing big frameworks from the ground up is very much a cornerstone of many agile development paradigms. And I have also seen what happens when you have a team design frameworks for 8 months, only to have unforeseen things happen ( that no one had any chance of predicting 8 months before ) make all that work worthless in an instant.
This is not me trying to be funny though. The point is that I could never in a million years have designed the Service Framework, and if you look at the history of the services and the framework, you will realize that I really didn't. What I did do was build one service, the Magnatune store, and then extrapolate the best parts of that ( the parts that really proved useful, and not the parts that I thought in advance would be useful ). I then used these parts to build the next service, Jamendo.com, and then extracted some more bits.
Doing development this way does cost you quite a few rewrites of stuff you have already done, but the benefit is that you don't start out by trying to look far into the future, trying to anticipate every possible feature that services some time in the future might require. This will in my experience only lead to you implementing tons of stuff that you do not really need, and missing bunch of features that you really should have been doing instead. By letting a framework evolve rather than designing it up front, it will likely become less complex and much better adapted to the stuff that it actually ends up being used for.
I should stop now. You sort of hit on of my push buttons
I am actually a certified scrum master and have worked quite extensively with lean and agile methodologies in one of my former jobs, and not designing big frameworks from the ground up is very much a cornerstone of many agile development paradigms. And I have also seen what happens when you have a team design frameworks for 8 months, only to have unforeseen things happen ( that no one had any chance of predicting 8 months before ) make all that work worthless in an instant.
There's actually a rule in game development which sort of fits this as well. It's a pretty simple one, which just reads "Don't write an engine, write a game" Coupled with your comment, it describes a problem found in some games pretty well
Coupled with your comment, it describes a problem found in some games pretty well
Both good points and I do agree that when doing something from the first time it serves to take small steps and revisit past code as new ideas emerge. Maybe that in itself is a lesson worth sharing?
It is, but I think others have done so way better than I will ever be able to
Personally I think someone like David Heinemeier Hansson (the guy who invented Ruby on Rails) explains it very eloquently ( from http://www.oreillynet.com/pub/a/network/2005/08/30/ruby-rails-david-heinemeier-hansson.html ):
"That's why we hold the notion that "frameworks are extractions" so very dear in the Rails community. Frameworks are not designed before the fact. They're extracted when you've proved to yourself that an approach works. Whenever we get ahead of ourselves and try to leap over the extraction process, we come back sorely disappointed."
Personally I think someone like David Heinemeier Hansson (the guy who invented Ruby on Rails) explains it very eloquently ( from http://www.oreillynet.com/pub/a/network/2005/08/30/ruby-rails-david-heinemeier-hansson.html ):
"That's why we hold the notion that "frameworks are extractions" so very dear in the Rails community. Frameworks are not designed before the fact. They're extracted when you've proved to yourself that an approach works. Whenever we get ahead of ourselves and try to leap over the extraction process, we come back sorely disappointed."
i hope that the "random" feature will be fixed by the release version... now isn't a great random between albums, it still in the same album... 
and by the 2.1 i hope to see again the filter and queue functionality
btw, great work and thanks for everything
and by the 2.1 i hope to see again the filter and queue functionality
btw, great work and thanks for everything
I am not really sure what you mean about the random feature.
Filter and queue will definitely be back again soon, as many devs hold these features very dear.
Filter and queue will definitely be back again soon, as many devs hold these features very dear.
I've add some albums to the playlist. With random=off when a track end it start the next one. But if i set random=tracks or album it's the same, when a track end it start just the next one following the order instead of select another track in a random way.
Another stuff i've noted is this: random:off, repeat: off, start a track, go to the end, it pass to the next one, while is playing set repeat:track, go to the end of the track, and it repeat itself...ok, for now is all right, but while is playing set repeat:off and go to the end of the track, it still repeat itself instead go to the next one. you have to stop and start the play manually.
I'm using Amarok Version 1.90 and KDE 4.1.3 (KDE 4.1.3) (compiled by the gentoo ebuilds), so maybe with the last beta these are already fixed
Another stuff i've noted is this: random:off, repeat: off, start a track, go to the end, it pass to the next one, while is playing set repeat:track, go to the end of the track, and it repeat itself...ok, for now is all right, but while is playing set repeat:off and go to the end of the track, it still repeat itself instead go to the next one. you have to stop and start the play manually.
I'm using Amarok Version 1.90 and KDE 4.1.3 (KDE 4.1.3) (compiled by the gentoo ebuilds), so maybe with the last beta these are already fixed
Ramdom works for me. If random=track, it will jump all over the playlist, if random=album, it will play one album to the end, and then play a random album from the start.
Do note that 1.90 is practically ancient, and all of these issues are most likely fixed either in Beta 3 or in current svn.
Do note that 1.90 is practically ancient, and all of these issues are most likely fixed either in Beta 3 or in current svn.
Ok, perfect
I'm still with the 1.90 because gentoo doesn't have the latest beta in portage (i don't know if they want to wait directly the release or the rc)
I'm still with the 1.90 because gentoo doesn't have the latest beta in portage
(i don't know if they want to wait directly the release or the rc)
The latest beta is in the kde-testing overlay, I had to keyword and unmask. This version fixed a lot of my problems.
I heard that there will be some basic video support in Amarok2. Is there any plan of creating a service for, say, PBS videos similar to NPR's audio library?
Yes, this will very likely be possible in later Amarok 2 versions.
In 2.0.0 we decided to disable the Video applet, as it is not quite finished yet.
In 2.0.0 we decided to disable the Video applet, as it is not quite finished yet.
Amarok Links
Calendar
Quicksearch
Categories
Syndicate This Blog
Blog Administration
We Listen To...
Powered by
