Tuesday, December 18. 2007
Windows binaries and packaging
No, I'm not announcing them, I'm explaining what is involved and why it's not just a matter of "throw up a zip file on a server".
Using things like kdelibs and ruby in Amarok is great: you get a lot of functionality that you don't need to develop yourself. On Linux someone else takes care of packaging it all up for you too, and since many programs use them, Amarok itself doesn't really pay the cost.
On Windows it's a bit different. From a quick test I did, a Visual Studio release build with the non-runtime stuff stripped away (debug libraries, linker files, header files, etc) comes out at around270mb 200mb. Compressed with 7zip it's a more reasonable 70mb, but it's still a fair bit of data to be pushing around if we're going to be doing a reasonably frequent alpha and beta release schedule. A lot of that is stuff Amarok doesn't need (eg base KDE programs and graphics), but stripping that out becomes pretty tricky, and I'm not sure we'd want to.
The good news is Amarok by itself itself comes out to only 2.35mb compressed: so assuming that KDE on Windows reaches a stable(-ish) point in the near future, putting up builds of just Amarok itself will be quite reasonable.
Packaging it up is also an issue. If we use the kde windows installer builds, then it causes me two problems: firstly that the release schedule might not be convenient for Amarok, and secondly that the compilers it supports are mingw and Visual Studio 2005. I'm primarily using Visual Studio 2008, and so I can't compile against the binaries from the installer. In theory I could release mingw builds, but at this stage the Visual Studio builds are smaller, faster and (at least for Amarok) better tested.
On the other hand, doing manual packaging means that Amarok wont be in sync with the rest of the kde stuff and that could cause issues if people want to use other KDE programs. Nobody really wants to contend with kde on Windows having a separate distribution for each program.
At the moment Amarok isn't far enough along that it's worth dealing with these issues. I'd rather spend energy on making it worth using than dealing with packaging and installation nightmares. Yay for Linux where someone else deals with that crap for you!
Update:
Some of my figures were wrong, it's "only" about 200mb in total.
A breakdown of where the space is going (in kb)
12488 amarok
75559 kdebase
33109 kdelibs
5155 kdepimlibs
134 kdewin32
158 qimageblitz
42188 qt
16984 ruby
1354 soprano
1336 strigi
352 taglib
21283 win32libs
In theory, at least some of the qt, kdelibs and kdepimlibs, and most of kdebase can be trimmed, with varying degrees of pain. Getting it down to maybe 120mb installed and a 35mb download would probably be the limit of what is easily achieved. Of course, as mentioned above, then you'd be left with a half broken KDE install that would probably screw up any other KDE app you attempted to install. I know some people wont care about any KDE apps other than Amarok, but for those who do, to break them would be really bad form.
Update 2:
I've started a thread on the kde-windows mailing list to see if we can work out how we're going to handle packaging and releasing KDE apps on Windows. Hopefully we can work something out. As I mentioned, we have an downloader/installer/packager app already, but it needs more work before it's ready to handle non-developer use.
Unfortunately KDE on Windows is still a fairly small project, and it's a bit of a catch 22: need more people to push it towards release, need a release to get people interested and involved.
As far as download size goes, a lot of it is about perception. The first impression you get of an application is how big the download is. I don't want Amarok coming off as "bloatware". Although I may know that the download includes mostly KDE related stuff that isn't Amarok (heck, you get a file manager, web browser, etc), a new user wouldn't and would just think "this is even more bloated than iTunes". Perhaps the best way to do this is getting interest in other KDE apps on Windows, so people can see they're getting a package that lets them download and run a whole lot of useful stuff. Again, it's a pity we don't have the manpower at the moment to do it.
Using things like kdelibs and ruby in Amarok is great: you get a lot of functionality that you don't need to develop yourself. On Linux someone else takes care of packaging it all up for you too, and since many programs use them, Amarok itself doesn't really pay the cost.
On Windows it's a bit different. From a quick test I did, a Visual Studio release build with the non-runtime stuff stripped away (debug libraries, linker files, header files, etc) comes out at around
The good news is Amarok by itself itself comes out to only 2.35mb compressed: so assuming that KDE on Windows reaches a stable(-ish) point in the near future, putting up builds of just Amarok itself will be quite reasonable.
Packaging it up is also an issue. If we use the kde windows installer builds, then it causes me two problems: firstly that the release schedule might not be convenient for Amarok, and secondly that the compilers it supports are mingw and Visual Studio 2005. I'm primarily using Visual Studio 2008, and so I can't compile against the binaries from the installer. In theory I could release mingw builds, but at this stage the Visual Studio builds are smaller, faster and (at least for Amarok) better tested.
On the other hand, doing manual packaging means that Amarok wont be in sync with the rest of the kde stuff and that could cause issues if people want to use other KDE programs. Nobody really wants to contend with kde on Windows having a separate distribution for each program.
At the moment Amarok isn't far enough along that it's worth dealing with these issues. I'd rather spend energy on making it worth using than dealing with packaging and installation nightmares. Yay for Linux where someone else deals with that crap for you!
Update:
Some of my figures were wrong, it's "only" about 200mb in total.
A breakdown of where the space is going (in kb)
12488 amarok
75559 kdebase
33109 kdelibs
5155 kdepimlibs
134 kdewin32
158 qimageblitz
42188 qt
16984 ruby
1354 soprano
1336 strigi
352 taglib
21283 win32libs
In theory, at least some of the qt, kdelibs and kdepimlibs, and most of kdebase can be trimmed, with varying degrees of pain. Getting it down to maybe 120mb installed and a 35mb download would probably be the limit of what is easily achieved. Of course, as mentioned above, then you'd be left with a half broken KDE install that would probably screw up any other KDE app you attempted to install. I know some people wont care about any KDE apps other than Amarok, but for those who do, to break them would be really bad form.
Update 2:
I've started a thread on the kde-windows mailing list to see if we can work out how we're going to handle packaging and releasing KDE apps on Windows. Hopefully we can work something out. As I mentioned, we have an downloader/installer/packager app already, but it needs more work before it's ready to handle non-developer use.
Unfortunately KDE on Windows is still a fairly small project, and it's a bit of a catch 22: need more people to push it towards release, need a release to get people interested and involved.
As far as download size goes, a lot of it is about perception. The first impression you get of an application is how big the download is. I don't want Amarok coming off as "bloatware". Although I may know that the download includes mostly KDE related stuff that isn't Amarok (heck, you get a file manager, web browser, etc), a new user wouldn't and would just think "this is even more bloated than iTunes". Perhaps the best way to do this is getting interest in other KDE apps on Windows, so people can see they're getting a package that lets them download and run a whole lot of useful stuff. Again, it's a pity we don't have the manpower at the moment to do it.
Trackbacks
Trackback specific URI for this entry
No Trackbacks
Comments
Display comments as
(Linear | Threaded)
Yes, it's a bit of a pain, when compared to Winamp's ~30mb or iTunes' ~120mb (between it and Quicktime). I guess it's better than WMP which is "part of the OS" and so counts as a multi-gb install, right?
A bit of playing around suggests we can probably get it down to somewhere around the iTunes mark if we prune unused icons, and more aggressively get rid of other things we don't need. It's a real pain to do that though and it's bound to break now and then (like everytime someone uses a new icon).
A bit of playing around suggests we can probably get it down to somewhere around the iTunes mark if we prune unused icons, and more aggressively get rid of other things we don't need. It's a real pain to do that though and it's bound to break now and then (like everytime someone uses a new icon).
Personally I would take about any size installer for Amarok just so that I could have it run under windows. Though I must admit that would be a bit discouraging to someone who is unfamiliar with Amarok itself.
Shouldn't windows users have the win32libs? and well if they have skype or something like that shouldn't they have the qt libs?
If I use kde in windows I don't think I will only use Amarok, I would also use at least kmail, k3b, kate..., and I wouldn't like to have problems with the kde libs. I think a lot of early adopters of amarok in windows will be kde users and I don't think that breaking the kde libs to make amarok installation smaller is the best thing
If I use kde in windows I don't think I will only use Amarok, I would also use at least kmail, k3b, kate..., and I wouldn't like to have problems with the kde libs. I think a lot of early adopters of amarok in windows will be kde users and I don't think that breaking the kde libs to make amarok installation smaller is the best thing
While it is true that some of these dependencies might already be installed, the lack of a package manager makes it difficult to do any sane form of dependency tracking and downloading if I understand the issue correctly.
- Nikolaj
- Nikolaj
The "win32libs" package is a collection of libraries that are dependencies of kde and you can rely on having under unix but aren't shipped with windows. Things like dbus, zlib, bzip, iconv, aspell, etc etc etc.
And we can't just rely on some other app's qt libs for a few reason:
- They might be the wrong version
- They might be compiled with the wrong compiler
- They might be compiled with the wrong compiler flags
- They'll almost certain have the wrong directory layout
- Even if all the above weren't a problem, how do we reliably find them? What if the user uninstalls the other app and breaks us?
And no, I don't think it's acceptable to break other KDE apps (I'm not actually suggesting that as a solution, more to point out we can save a little more space but we wouldn't want to do it). If every app decides to only look out for itself it's a nightmare.
And we can't just rely on some other app's qt libs for a few reason:
- They might be the wrong version
- They might be compiled with the wrong compiler
- They might be compiled with the wrong compiler flags
- They'll almost certain have the wrong directory layout
- Even if all the above weren't a problem, how do we reliably find them? What if the user uninstalls the other app and breaks us?
And no, I don't think it's acceptable to break other KDE apps (I'm not actually suggesting that as a solution, more to point out we can save a little more space but we wouldn't want to do it). If every app decides to only look out for itself it's a nightmare.
I agree with you, I really don't mind about a 200Mb download if it gives me amarok
The download is only 70 in the worst case scenario, 200 is the decompressed size. 
Size doesn`t matter. I want it. What for is fast internet? I would download it even if it is 2 Gigs...
what about pidgeon? those guys managed to run it both on windows and linux and the windows installer did not have monumental sizes?
Pidgin used GTK not QT as a graphical toolkit and is written in just C.
Most of the size comes from KDE. Which is understandable, for what KDE aims to do they need it.
Hmm... How about splitting the install into two "packages": Let's call one of them "Linux compatibility package" (we can spend time on finding a better name later on) and the other "Pure Amarok".
Disadvantage:
- Might confuse users. However... I think most of the "first Amarok on Windows" users will be the ones who know it from Linux...
Advantages:
+ People realize that most of the download is needed to provide the foundation for the program, not the program itself.
+ An Amarok-"Update" (esp. nightlies, alphas, betas etc.) can be rather small.
Just a thought...
Disadvantage:
- Might confuse users. However... I think most of the "first Amarok on Windows" users will be the ones who know it from Linux...
Advantages:
+ People realize that most of the download is needed to provide the foundation for the program, not the program itself.
+ An Amarok-"Update" (esp. nightlies, alphas, betas etc.) can be rather small.
Just a thought...
Have an option in the installer (which will only be the "pure amarok") which checks if you already have the required KDE files installed, and if not will download and install them for you
Why not use static linking? Then the optimizing compiler will trim the unneeded stuff with no effort on your part.
You can't statically link graphics.
And a lot of stuff relies on dynamic linking with the way KDE works.
And a lot of stuff relies on dynamic linking with the way KDE works.
Technically they could be included using qrc. In practice, though, that would need heavy patching throughout the code, just as statically linking all code would. (I don't think either is really practicable. As you said, a lot of stuff relies on dynamic linking.)
IIRC, at least on Ubuntu people who use gnome just need to install kdelibs and the non-KDE dependencies. Especially, there is no need for all these apps which are part of KDE. So why do people who use win32 have to install kdebase? Stripping it out should already greatly reduce the package's size. Since kdelibs is itself composed of lots of small libraries, one should be able to strip the unneeded libraries out. If it works for gnome, it should also work for win32 pretty easily.
Another point is that breaking all other KDE apps by stripping most parts of it out is't a problem anyway because you could provide both a special sandbox-like edition with only these parts which are vital to Amarok and a part-of-KDE solution which resides in a web-installer that is able to track dependencies itself. This way you provide solutions for both Amarok-only people and KDE-win32 users.
Another point is that breaking all other KDE apps by stripping most parts of it out is't a problem anyway because you could provide both a special sandbox-like edition with only these parts which are vital to Amarok and a part-of-KDE solution which resides in a web-installer that is able to track dependencies itself. This way you provide solutions for both Amarok-only people and KDE-win32 users.
We need kdebase or else you get no icons or sound. Kind of breaks the app without it. Since the phonon backend is in kdebase, you'll probably want to install kdebase on linux for Amarok 2.0 as well.
And you can't sandbox a KDE install, due to things like dcop. Two sandboxed programs wont run happily together, they have to run from the same installation.
And you can't sandbox a KDE install, due to things like dcop. Two sandboxed programs wont run happily together, they have to run from the same installation.
The icons thing must have changed from KDE3 to KDE4. If it's like that, you could probably change it by putting the icons in an extra part. Phonon is now in Qt, isn't it? You could link it from there
I'm not that much into the new structures, so don't believe I know that for real
I'm not that much into the new structures, so don't believe I know that for real
Make available two solutions.
Easy install: Provides the minimal kde libraries for Amarok to work. The libraries will be installed in the same directory as Amarok since they will NOT work for other kde apps. Newbie's installation.
Recommmended install: Two separated packages. Amarok + KDE base(may be downloaded from other sites. i.e. koffice). Intended for users who use other kde apps. Experienced users.
Easy install: Provides the minimal kde libraries for Amarok to work. The libraries will be installed in the same directory as Amarok since they will NOT work for other kde apps. Newbie's installation.
Recommmended install: Two separated packages. Amarok + KDE base(may be downloaded from other sites. i.e. koffice). Intended for users who use other kde apps. Experienced users.
Why isn't there a separate kdebase-runtime package, with only the stuff in the kdebase-runtime tarballs (i.e. the kdebase/runtime directory in KDE SVN)? This should solve the "Why do I have to install Konqueror to use Amarok?" type issues. It's also what most the GNU/Linux distributions do. (At least the Fedora packages I'm comaintaining do that.) You'll also need libplasma packaged in some way (kdebase-workspace-libs?) though.
You can try to find out how the guys fom last.fm (5MB) and musicbainz (6MB) managed to get the sizes stripped.
As far as I know both of the apps use (ro used) QT:
http://blog.musicbrainz.org/archives/2007/10/picard_qt_enter.html
http://www.last.fm/download/
As far as I know both of the apps use (ro used) QT:
http://blog.musicbrainz.org/archives/2007/10/picard_qt_enter.html
http://www.last.fm/download/
I think the most important thing is getting a stable build, then we can worry about the size
I agree.
Some issues will probably solve themselves, eg Qt 4.4 will have the Phonon backend for Windows so only the icons from kdebase/runtime will really be needed.
Some issues will probably solve themselves, eg Qt 4.4 will have the Phonon backend for Windows so only the icons from kdebase/runtime will really be needed.
As long as the KDE libs end up n "\Program Files\Common Files\KDE" and not Amarok's folder, it shouldn't be a huge problem. Especially if the user also has KOffice 2 installed and both use the same files and do not end up with two duplicate copies of their dependencies in their own folders.
The install layout is unix-like: amarok.exe installs into %KDEROOT%/bin, with various bits in %KDEROOT%/share, etc.
You could just make the smaller amarok download as the only one that is really visible on the website. If the installer is run and it doesn't find the correct version of kdelibs, it can then prompt the user with a little background info and then download and install kdelibs first.
Ugh, I was kind of forced to move back to Windows after an awesome year and a bit of Linux. And I really really miss Amarok. It did everything I wanted, except multiple queuing of the same track. I really really hope a Windows Amarok would be available soon -- It's hard for me to get along without music organizing, album art managing, and lyrics searching, and most importantly queuing. I'm using iTunes right now, and I can't queue tracks, and shuffle just mixes the playlist and makes it harder to find songs, instead of just playing songs randomly out of the organized playlist.
I cannot find amarok for windows anywhere. Could somebody post a link to it?
You can't find it because it hasn't been released yet.
meh windows users cant complain alot of the MS programs require you to install their crap: .net etc... people will just have to realize how good the amarok is then they will move over to it.
Amarok Links
Calendar
Quicksearch
Categories
Syndicate This Blog
Blog Administration
We Listen To...
Powered by
