In the Precise development cycle a new archive oriented team was trialled, the Plus One team. This team was created to help keep the archive in an installable and buildable state at all times, a major driver for early testing and for a solid LTS release. The Plus One team is tasked with general house keeping for the Ubuntu archive. for finding issues such as build failures or packages which are no longer built from the source in the archive, and figuring out what to do with them. They are also responsible for figuring out why our images are broken on any particular day and driving resolution.
This cycle when they were looking for volunteers for the team for the Quantal cycle I was put forward to help. It sounded interesting as the work has a much broader base than my normal role, touching anything in the Ubuntu archive. I have been working with Debian packages for over 4 years, but mostly kernel related packages and they have their own quirks. This gave me a chance to branch out and solidify my Ubuntu skills, a good stepping stone to becoming a core-dev in my own right. An exciting and scary prospect.
I have been working on the Plus One team for a couple of weeks now, and all I can say is it has been a baptism of fire, have had to touch C++, C, python, ruby, perl (and more) often in the same package. Had to fiddle with autoconf, and get familiar with the multitudinous patching schemes. I have learned a healthy hatred for quilt (as it lets me lose my changes for the umpteenth time). Test built untold packages; my poor test build server is crying out for a rest after the utter pounding I have given it. For all this work we have made some progress indeed, but at this time in the cycle the breakage is building at least as fast as we club it into submission. At times it is soul destroying.
Overall though it has been a very positive experience, I have learned a huge amount about the archive and packaging in the Ubuntu world, and gained a healthy dose of respect for anyone who voluntarily maintains other peoples packages. All I can really say is a big thank you to those who look after this stuff full time, you are made of strong stuff indeed.
Wednesday, 20 June 2012
Monday, 28 May 2012
World IPv6 Launch Day -- 6th June 2012
The 6th of June 2012 is an interesting day, World IPv6 Launch Day. On this day a swathe of influential web-sites will be enabling IPv6 addresses for their services by default and not turning them off. Why is this interesting? For the most part it is not! For most people nothing should happen, things should continue working probably using IPv4, fine. For those of us with working IPv6 again nothing will change other than we will be producing more IPv6 traffic, great. It is those who have unused broken IPv6 who will start having fun, they will likely lose connectivity to the participating sites until they sort out their issues.
So why is this a good thing? This is a key step towards IPv6 adoption, and we really do not have any other choice but to adopt IPv6. Adoption will only be driven by need, and this move creates exactly that need. Such ISP and client issues need to be identified and solved, before the end users lose connectivity to even greater swathes of The Internet, those parts which only will exist on The IPv6 Internet.
For more information see: http://www.worldipv6launch.org/
So why is this a good thing? This is a key step towards IPv6 adoption, and we really do not have any other choice but to adopt IPv6. Adoption will only be driven by need, and this move creates exactly that need. Such ISP and client issues need to be identified and solved, before the end users lose connectivity to even greater swathes of The Internet, those parts which only will exist on The IPv6 Internet.
For more information see: http://www.worldipv6launch.org/
Tuesday, 15 November 2011
IPv6 at home?
The Internet has been alive with doom saying since the IPv4 global address pool was parcelled out. Now I do not subscribe to the view that the Internet is going to end imminently, but I do feel that if the technical people out there do not start playing with IPv6 soon then what hope is there for the masses?
In the UK getting native IPv6 is not a trivial task, only one ISP I can find seems to offer it and of course it is not the one I am with. So what options do I have? Well there are a number of different types of IPv4 tunnelling techniques such as 6to4 but these seem to require the ability to handle the transition on your NAT router, not an option here. The other is a proper 6in4 tunnel to a tunnel broker but this needs an end-point.
As I have a local server that makes a sensible anchor for such a tunnel. Talking round with those in the know I settled on getting a tunnel from Hurricane Electric (HE), a company which gives out tunnels to individuals for free and seems to have local presence for their tunnel hosts. HE even supply you with tools to cope with your endpoint having a dynamic address, handy. So with an HE tunnel configuration in hand I set about making my backup server into my IPv6 gateway.
First I had to ensure that protocol 41 (the tunnelling protocol) was being forwarded to the appropriate host. This is a little tricky as this required me to talk to the configurator for my wireless router. With that passed on to my server I was able to start configuring the tunnel.
Following the instructions on my HE tunnel broker page, a simple cut-n-paste into /etc/network/interfaces added the new tunnel network device, a quick ifup and my server started using IPv6. Interestingly my apt-cacher-ng immediately switched backhaul of its incoming IPv4 requests to IPv6 no configuration needed.
Enabling IPv6 for the rest of the network was surprisingly easy. I had to install and configure radv with my assigned prefix. It also passed out information on the HE DNS servers, prioritising IPv6 in DNS lookup results. No changes were required for any of the client systems; well other than enabling firewalls. Win.
Overall IPv6 is still not simple as it is hard to obtain native IPv6 support, but if you can get it onto your network the client side is working very well indeed.
In the UK getting native IPv6 is not a trivial task, only one ISP I can find seems to offer it and of course it is not the one I am with. So what options do I have? Well there are a number of different types of IPv4 tunnelling techniques such as 6to4 but these seem to require the ability to handle the transition on your NAT router, not an option here. The other is a proper 6in4 tunnel to a tunnel broker but this needs an end-point.
As I have a local server that makes a sensible anchor for such a tunnel. Talking round with those in the know I settled on getting a tunnel from Hurricane Electric (HE), a company which gives out tunnels to individuals for free and seems to have local presence for their tunnel hosts. HE even supply you with tools to cope with your endpoint having a dynamic address, handy. So with an HE tunnel configuration in hand I set about making my backup server into my IPv6 gateway.
First I had to ensure that protocol 41 (the tunnelling protocol) was being forwarded to the appropriate host. This is a little tricky as this required me to talk to the configurator for my wireless router. With that passed on to my server I was able to start configuring the tunnel.
Following the instructions on my HE tunnel broker page, a simple cut-n-paste into /etc/network/interfaces added the new tunnel network device, a quick ifup and my server started using IPv6. Interestingly my apt-cacher-ng immediately switched backhaul of its incoming IPv4 requests to IPv6 no configuration needed.
Enabling IPv6 for the rest of the network was surprisingly easy. I had to install and configure radv with my assigned prefix. It also passed out information on the HE DNS servers, prioritising IPv6 in DNS lookup results. No changes were required for any of the client systems; well other than enabling firewalls. Win.
Overall IPv6 is still not simple as it is hard to obtain native IPv6 support, but if you can get it onto your network the client side is working very well indeed.
Tuesday, 21 June 2011
Oh no 3.0
After 39 2.6.x releases Linus Torvalds has chosen to revisit the upstream kernel version. The plan is to release what would have been 2.6.40 instead as version 3.0:
After much discussion we decided we bite the bullet and upload a 3.0 kernel. At least we get a chance to identify problematic applications, while still keeping our options open to move to a 3.0.0 kernel for release should that be prudent. As expected this was not smooth sailing, not least for the kernel packaging which needed much love to even correctly build this version. Plus we had to hack the meta packages to allow that to be reversioned later too.
Once successfully uploaded the problem applications started to crawl out of the woodwork:
"I decided to just bite the bullet, and call the next version 3.0. ItWhen 3.0-rc1 was released the Kernel Team had to decide what version to use for it in Ubuntu. We typically upload every -rcN release within a couple of days of its release so the pressure was on. We could simply call it 3.0.0 knowing that all the current scripting would cope, or as 3.0 better matching its official name knowing this would not be plain sailing. This was not a decision we could delay as in Debian versioning 3.0 < 3.0.0 so we were likely to be committed for Oneiric if we uploaded using 3.0.0. It is also not clear from upstream discussion what version number the final release will carry, as 3.0 clearly will cause breakage on older userspace.
will get released close enough to the 20-year mark, which is excuse
enough for me, although honestly, the real reason is just that I can
no longe rcomfortably count as high as 40."
After much discussion we decided we bite the bullet and upload a 3.0 kernel. At least we get a chance to identify problematic applications, while still keeping our options open to move to a 3.0.0 kernel for release should that be prudent. As expected this was not smooth sailing, not least for the kernel packaging which needed much love to even correctly build this version. Plus we had to hack the meta packages to allow that to be reversioned later too.
Once successfully uploaded the problem applications started to crawl out of the woodwork:
- depmod -- the depmod incantion to create the module dependancies identifies the kernel version in its command line but was assuming that a version contained three digits, this lead it to miss the version entirely and rebuild the wrong dependancies;
- libc6 -- both the runtime and the installation control scripts manipulate the kernel version number, in both cases assuming the version was three digits, enormous fun getting the pending updates installed;
- ps/top -- when starting the kernel version was checked, and miss decoded triggering a rather nasty sounding version warning whenever they are started;
- nfs-utils -- when attempting to read and identify the kernel version the nfs-utils would trigger a SIGSEGV and die, triggering boot failures on machines with NFS roots; and
- lm-sensors-3 -- this package is only compatible with 2.6.5 and above, failed version detection lead to this test failing and sensors being unconfigured.
Those are the ones we have found so far, I am sure there will be more. If you do find one please file a bug against the failing package but tag it kernel-3.0 then we can find them.
Thursday, 19 May 2011
Union File Systems (again)
During the early part of the Maverick cycle we once again revisited out Union Mount solution. At that time VFS union-mounts was the hit of the day, set to finally to produce something which might get into the kernel. Since then the complexity of changing every filesystem to support whiteouts, its invasiveness, and its affects on Posix semantics have lead to it falling by the wayside. In its place has sprung overlayfs.
overlayfs is a small patch set which is a hybrid of the VFS union-mount approach and that of aufs/unionfs in that it also provides a filesystem. This greatly reduces the complexity of the patch set, reducing its invasiveness and thus increasing its chances of ever being merged. So much so simpler is it that your author is actually able to understand and debug it. Win.
We have been tickling overlayfs for most of the Natty cycle, but with Natty in the can I have had had some time to catch up with its development and help out a little, both with testing and bug fixing. Culminating today in my being able to inject a kernel containing overlayfs support into an Ubuntu LiveCD and boot it, then update it to the latest Natty, all without error.
overlayfs may shortly be in a mergable state, nirvana for all union mount lovers. Only time and testing will tell.
overlayfs is a small patch set which is a hybrid of the VFS union-mount approach and that of aufs/unionfs in that it also provides a filesystem. This greatly reduces the complexity of the patch set, reducing its invasiveness and thus increasing its chances of ever being merged. So much so simpler is it that your author is actually able to understand and debug it. Win.
We have been tickling overlayfs for most of the Natty cycle, but with Natty in the can I have had had some time to catch up with its development and help out a little, both with testing and bug fixing. Culminating today in my being able to inject a kernel containing overlayfs support into an Ubuntu LiveCD and boot it, then update it to the latest Natty, all without error.
overlayfs may shortly be in a mergable state, nirvana for all union mount lovers. Only time and testing will tell.
Friday, 2 July 2010
Ubuntu Kernel Crack of the Day
We have been producing the mainline kernel crack of the day[1] for some time now. But this targets the upstream kernel, and while great for testing and bug isolation it does not provide us with any pre-upload testing on the Ubuntu kernel delta.
Enter the pre-proposed kernel PPA[2]. We are now uploading the unreleased tips of the ubuntu kernel trees to this PPA. These builds will contain any bug fixes marked Fix Committed and should provide a vehicle for advanced testing of these before they hit the archive. For Maverick we will be uploading these automatically as the tree changes, roughly daily. This should help us avoid the ThinkPad debackle we experienced late in Lucid.
I would encourage you to add this PPA to your sources and help us test this kernel before we unleash it on the world.
[1] https://wiki.ubuntu.com/KernelTeam/MainlineBuilds
[2] https://edge.launchpad.net/~kernel-ppa/+archive/pre-proposed/
Enter the pre-proposed kernel PPA[2]. We are now uploading the unreleased tips of the ubuntu kernel trees to this PPA. These builds will contain any bug fixes marked Fix Committed and should provide a vehicle for advanced testing of these before they hit the archive. For Maverick we will be uploading these automatically as the tree changes, roughly daily. This should help us avoid the ThinkPad debackle we experienced late in Lucid.
I would encourage you to add this PPA to your sources and help us test this kernel before we unleash it on the world.
[1] https://wiki.ubuntu.com/KernelTeam/MainlineBuilds
[2] https://edge.launchpad.net/~kernel-ppa/+archive/pre-proposed/
Tuesday, 1 June 2010
Union Filesystem Plans
At UDS we return to the subject of those drivers we are carrying which are specific to Ubuntu, why they are not yet upstream and what we can do about it. Union Filesystems are a key technology in producing the live CD environment used to allow both non-destructive testing and the graphical installer. This technology has long been a contentious subject as the patch sets have been extensive and intrusive to the VFS. Worse there have been a number of camps all disagreeing as to the most sensible solution.
For a number of series we have carried the AUFS/AUFS2 patch kit as an Ubuntu add on. This has been a solid performer in this space and served our purposes well. About a year ago now talk began upstream on what approach would be acceptable to upstream. This has resulted in a proposed for an integrated VFS based solution for union filesystems called union-mounts. Patches have been circulating for the best part of a year and we skirted with them for the Lucid cycle, at that time they were feature incomplete preventing full testing.
Recently updated patches have been circulated which should be feature complete, and we are planning to provide kernels enabled for union-mount for testing in a PPA. Should testing there prove good we will consider switching to this solution for our live CDs. Watch this space as they say.
For a number of series we have carried the AUFS/AUFS2 patch kit as an Ubuntu add on. This has been a solid performer in this space and served our purposes well. About a year ago now talk began upstream on what approach would be acceptable to upstream. This has resulted in a proposed for an integrated VFS based solution for union filesystems called union-mounts. Patches have been circulating for the best part of a year and we skirted with them for the Lucid cycle, at that time they were feature incomplete preventing full testing.
Recently updated patches have been circulated which should be feature complete, and we are planning to provide kernels enabled for union-mount for testing in a PPA. Should testing there prove good we will consider switching to this solution for our live CDs. Watch this space as they say.
Subscribe to:
Posts (Atom)