Services upgrade and database prune

Hi
The long-awaited upgrade of services which we blogged about a while ago is now planned for this coming weekend, the 16th/17th June.

We anticipate up to an hour of services outage for this upgrade and prune to take place. We will notify the connected users closer to the time through the use of WALLOPS and/or globals, but please do plan ahead accordingly for a period of services unavailability.

We will be moving to Atheme 7, so, amongst other improvements, this will see the introduction of certificate-based authentication to services.

To use certificate based authentication, you would need firstly to generate a certificate, then add the certificate to your client, then tell nickserv about your certificate fingerprint. We’ll explain more about this in a future blog entry or on the freenode website in the near future.

Database prune

Every couple of years, freenode likes to get out the shears and prune the services database. Recently we broke the 80,000 usercount barrier, but the services stats are way ahead:

Sat 13:35:46 -OperServ(OperServ@services.)- Registered accounts: 446777
Sat 13:35:46 -OperServ(OperServ@services.)- Registered nicknames: 557497
Sat 13:35:47 -OperServ(OperServ@services.)- Registered channels: 141373

We’ve noticed that nearly half of the accounts shown there haven’t been used in the past 6 months! More importantly, over the past few months many people have noticed significant waits when issuing certain services commands – and we’d like to fix that.

Hopefully, the services upgrade should help with this, but we’re going to coincide this with a database prune.

As of the services upgrade date, any nicks unused for > 150 days are at risk of being dropped. This includes grouped nicks. The easy way to avoid this happening is to use each of your grouped nicks (while identified to the appropriate account) within the next few weeks – and to drop those that you don’t need anymore!

The testnet (testnet.freenode.net, port 9002. 9003 for SSL) is running a database snapshot from mid-March and will be periodically updated from the production network. This database instance is being regularly pruned – so check there to see how your account will be affected (use /msg nickserv info on both the production and test networks to see the differences).

Remember that testnet isn’t running a real-time duplicate of the production network, so when you use nicks which would be expired on the production network, they will still appear expired on testnet until the next database snapshot is migrated. Don’t worry though – the actual pruning will only occur on the current database at the time of upgrade.

On which note.. an upgrade date hasn’t been formally fixed but we’re aiming for mid-May.

Thanks, and don’t forget to test the testnet!

Help us test our services upgrade!

Very soon we will be upgrading your favourite network helpers… (no not erry…): NickServ, ChanServ, Alis etc. They’re currently connected to our testnet and we need your help with testing, looking for any issues which may affect the production network.

You can connect to our testnet at testnet.freenode.net port 9002 (or 9003 for SSL)

The full changelog is rather long and not all of the features offered by atheme are loaded on freenode. So to help you out, we’ve pulled out the highlights which we think deserve attention:

  • NickServ’s certfp module. (see /msg nickserv help cert and this link.)
  • NickServ will now notify you in real time of failed logins.
  • NickServ’s previous limit on password lengths has been increased.
  • ChanServ will still hand over single-# channels to freenode-staff on expiration of the channel founders, but the method has changed.
  • NickServ & ChanServ’s ‘set’ commands have had a general reorganisation behind the scenes. Nothing should be visibly different but it won’t hurt to check them!

Please note that the services database on the testnet is probably more than a few days old. Don’t be surprised if recent changes you have made on the production network aren’t replicated there.

We’re all in in #freenode on the testnet so please come find us there if you have any questions or bugs.

Finally, look out for a followup blogpost (hopefully quite soon) with some important information on the upgrade itself and our planned database cleanup!

Thanks for using freenode!

P.s. a full list of changes from atheme ~5.1 to ~6 can be found here

80k…

As of today, freenode had 81443 concurrent connections. Last Friday, the count lingered around 79987, presumably to make things more interesting for those of us keeping close track.

We passed this milestone one working day later than I predicted in January 2011, more than a year ago. Scary precision, I know. Extrapolation based on graphs and nothing else is a big no-no in the scientific circles, yet we really do grow at an exceptionally steady and predictable pace.

Still remembering the excitement when irc.openprojects.net surpassed 2,000 users, I have always had a knack for those numbers. Meaningless in and as of themselves, they do prove that we are able to provide a valuable service to our users and that more and more people apparently agree. freenode is not without growing pains, but I like to think that we are on a good path.

Once again, all of us would like to thank all our users and all our volunteers. It’s a fun ride for us and, hopefully, for you, too!

To make tracking our old posts regarding this easier, here goes:

http://blog.freenode.net/2007/08/freenode-has-reached-40-000-users/

http://blog.freenode.net/2008/09/50000-active-users/

http://blog.freenode.net/2009/12/happy-new-year-2010/

http://blog.freenode.net/2011/01/freenode-70k/

See you around July 2013 when we expect to hit 90k.. :)

Merry Christmas

It was the day before Christmas, when all across the network not a creature was stirring, not even a troll…

Another year is coming to a close, a year of FOSS, a year of collaboration, a year of getting to know exciting projects and contributors, a year plentiful in conferences and events. A year in which community was at the heart of everything freenode and the PDPC did.

I am sat here in a candle lit room that smells of Christmas spice whilst sipping eggnog and stealing a moment to myself to reflect… when I first started using freenode, the network had around 1200 users — some years on we’ve passed 76,000. It is great to see so many projects make use of the service! The PDPC is doing some exciting stuff, FOSSCON was arranged for the second year running in 2011, FOSSEVENTS is still going strong and Geeknics are being held across the globe. Sometimes it is a bit daunting, everyone involved with freenode (and other PDPC projects) volunteer their time and skills to help the communities — and whilst this is great, the volunteer roster isn’t growing at a rate matching the increase in users. We will be doing another “Call for volunteers” in the New Year and we’re looking to find some awesome people to add to the team.

On a more personal basis, I’ve had the pleasure of participating in some brilliant “real life” events this year – from the always brilliant OGGCAMP, arranged by the amazing people behind the Linux Outlaws and Ubuntu UK podcasts. OGGCAMP is the sort of event which attracts “my sort of geek” — people with which I really enjoy spending time and socialising and I am already looking forward to next years! I’ve also enjoyed a number of Ubuntu-UK events, from the recent release party to smaller happy hours to the rather interesting Christmas meal at Dans Le Noir, where we ate a surprise menu (consisting of glow in the dark scallops, ostrich and blue shark to name some items!) in complete darkness before wandering off to enjoy a few pints of ale. Now, this is where freenode communities are great — I am an avid fan of the Ubuntu project, however, I have never used the distribution (though as a Debian user I am sure I’d get on with it just fine.. right?) but through freenode I have, over the years, come to know a lot of Ubuntu contributors and users, and over time discovered that these are people I really enjoy spending my time and people I am proud to call my friends! And I love their social events, they are a lot of fun! So thank you for letting me be a part of your community despite not really being “one of you” ;)

On the subject of freenode, communities and Christmas parties — as freenode volunteers are scattered across the globe, few of us meet on a regular basis but we tend to try make an effort to get together for dinner and drinks at events such as fosdem. This year, we decided to have a Christmas party, surprisingly we ended up with 21 people attending, most of which had never met each other in person before. Our volunteers flew over from America and Europe and we all had a smashing weekend filled with good food, nice drink and much laughter. Thank you all for coming!

In 2012 I look forward to learning about more projects, old and new. I look forward to attending yet more conferences and meeting more of you in person! I look forward to another year together with our volunteers and our sponsors, and of course our users.

On which note I shall wrap this entry up, thank you for using freenode and wish you all a Merry Christmas!

 

 

ircd upgrades

We’ve got some ircd upgrades in the works!

You may remember several weeks ago that we upgraded our ircd on the production network. Since then, we’ve wanted to fine-tune some changes and make sure that the upgrade is more consistent with the old version.

Over the next few weeks, we’ll be looking to perform upgrades on the production network again. This will mean every server will reboot. A programme for the upgrades can be found at the end of this post (updated 13th Nov 2011).

In the meantime, please continue to help us to test the ircd at testnet.freenode.net port 9002 or 9003 for SSL (if you don’t get onto the first server that the DNS roundrobin gives you, keep trying!). Look for anything broken, inconsistent with previous versions (especially in terms of information release) or illogical. If serious issues are reported, we’ll aim to fix before upgrading, rather than having a further later upgrade. Please report issues to #freenode-seven on the production network.

Thanks!

NB: this list does not include servers invisible to users (eg hubs).
Week 1: Sun 13th Nov
-!- kornbluth.freenode.net Frankfurt, Germany
-!- zelazny.freenode.net Corvallis, OR, USA
-!- stross.freenode.net Corvallis, OR, USA (webchat backup)

Week 2: Sun 20th Nov
-!- barjavel.freenode.net Paris, FR
-!- wolfe.freenode.net Manchester, England
-!- hubbard.freenode.net Pittsburgh, PA, US

Week 3: Sun 27th Nov
-!- adams.freenode.net Budapest, HU, EU
-!- holmes.freenode.net London, UK
-!- sendak.freenode.net Vilnius, Lithuania, EU
-!- rowling.freenode.net Corvallis, OR, USA (webchat)

Week 4: Sun 4th Dec
-!- pratchett.freenode.net Rennes, France
-!- calvino.freenode.net Milan, IT
-!- leguin.freenode.net Ume?, SE, EU
-!- niven.freenode.net Corvallis, OR, USA

Week 5: Sun 11th Dec
-!- hitchcock.freenode.net Sofia, BG, EU
-!- gibson.freenode.net Oslo, Norway
-!- card.freenode.net Washington, DC, USA
-!- asimov.freenode.net TX, USA
-!- verne.freenode.net Newark, NJ, US

Unscheduled.
-!- roddenberry.freenode.net
-!- bartol.freenode.net
-!- brown.freenode.net
-!- anthony.freenode.net


Update: all upgrades are now complete.

New GRF-freenode process

As you might know, GRFs (Group Registration Forms) exist to form a relationship between a project and the PDPC (Peer Directed Projects Center). This relationship is relatively formal – personal details (address/tel no./etc) need to be shared by the project. For this reason, a severe backlog of GRFs has built up, since only a few staff have access to them (to protect this personal data). PDPC is the UK-based not-for-profit company which runs freenode. For most groups in our request backlog, their reason for registering is not to work with PDPC, but to gain a channel or cloak namespace on freenode. We’ve decided that running a separate, freenode-centric groups request system may help to move the system along. By requesting fewer details, we can open up this system to more staff, and hopefully keep on top of the queue of requests.

From now on, using a new, shorter form, projects can choose to file a GRF-f (for GRF-freenode) and submit a GRF for processing by freenode, rather than by PDPC. This sends details (no personal details, other than email address, will be required) to a system to which many more staff will have access. This new form will allow you to gain control of a channel and the right to issue cloaks much more quickly than previously, as we will double/treble the number of staff able to deal with requests. For now, please only apply if you are a ‘priority’ group – ie, you do not own the main channel of your namespace.

If you already have a group registered and approved with an old-style GRF, you do not need to do anything. Your registration remains valid. If you need to make changes to the registration, please contact staff on freenode who will, if appropriate, direct you to use the old (GRF) system. The GRF-f system cannot be used to update groups which filed under the GRF system.

If you have a request pending in the old GRF queue, you are welcome to re-file under the GRF-f system. This is likely to mean that your request will be dealt with much more quickly than otherwise. This approach supersedes the grfprocess@ system introduced a while back – unfortunately, we just weren’t able to keep up with requests to that address.

You might be wondering where all of this fits into the GMS (Groups Management System) masterplan. When GMS is ready, we may need to ask all projects registered under the GRF-f system, and likely some projects which are already registered, to re-file. The GMS system will allow us to dispense with GRF-fs, and just build project<=>PDPC relationships, since forms will be able to be processed much more quickly. To be clear – it is quite possible that any registration made now may be revoked if a registration is not re-filed after GMS is released. If this does become reality, as much warning as possible will be given.

We hope that this will change will counter some of the ill-feeling around the GRFs system. In effect, the mentality is shifting from one of “GMS will clear the GRFs backlog” to “GMS will help us to serve groups better”. We’re no longer waiting for GMS to clear the queue. We’re still looking for help with GMS: if you have Perl/Catalyst or web design experience and think you can help, join #freenode-gms.

Update 2012-06-09 – All group registration has been suspended whilst we evaluate the system and its implementation. A replacement should be available in due course, but for now it is not possible to register groups, and the link to the grf-f form has been removed.

A short mention on Mibbit

Hi all.

It has come to our attention that Mibbit have experienced a security issue impacting (or so it initially appears) their testing servers. This has resulted in a list of plain text NickServ passwords from Mibbit’s own NickServ (the one you use if you connect to irc.mibbit.com) being made public, as well as a small section of private logs and some internal server details of the affected equipment. There’s some more information on their own blog.

As I’m sure many of you have been aware, Mibbit hasn’t been available for use with freenode for quite a while now so the issue will only indirectly impact freenode users when the password they use for their NickServ is the same as it is for elsewhere.  Therefore freenode staff are recommending that anyone that has used Mibbit’s own NickServ should ensure that their password is changed to keep your account secure.

Of course, it’s always good practice to change passwords periodically so you may wish to take the opportunity to do so as a matter of course.

Sponsorship Roundup

As you may know, the network operations of freenode are fully supported by donations – of hosting and other resources – from both companies and individuals. We acknowledge all sponsors on our website, but it is nice from time to time to provide a round-up of recent changes on the sponsorship scene!

If you’re currently connected to freenode, you will be connected to a donated server – look at the “MOTD” (delivered to you on connection or by passing the command /motd) to see who has provided your particular server.

Our newest servers include roddenberry.freenode.net (Brisbane, Australia) and asimov.freenode.net (Dallas, Texas, USA), provided by On Q Telecom and by Rackspace, respectively.

Worthy of mention indeed are those companies who support the network in ways other than providing servers. Gandi provides our SSL certificate and acts as our domain registrar, and Simtec Electronics recently generously supported the network with a donation of entropykeys. Look out for a later technical blog post as we roll these out!

While this post focuses on recent additions to the sponsorship team, it’s important not to forget the ongoing contributions of all our sponsors – take another look at our acknowledgements section and give these groups the kudos they deserve!

Cheers,
Martin