Server hosting and trust

For the purpose of disclosure we have had to make the difficult decision to discontinue a long-standing relationship with a server sponsor.

As a freenode user you may be aware that our set-up is somewhat untraditional and differs from that of many other IRC networks; servers are sponsored by various companies and educational institutions across the globe and all our infrastructure is centrally managed by the freenode infrastructure team. Generally speaking we do not provide o:lines or other privileges to server sponsors. Whilst it is possible for a sponsor contact to also volunteer as a staffer on the network such recruitment is independent of any server hosting.

Our staff are expected to work together closely and communication is key in any freenode relationship, be that with users, among staff or with sponsor contacts. It is important to us to be consistent in the way we provide support and apply policy and we expect all volunteers to be intimately familiar with our policies, procedures and philosophies — which in turn means that senior staff invest a lot of time in ensuring that any new recruits are given adequate support when getting to know the ins and outs of the network and what being a freenode volunteer entails.

Unfortunately one of our server sponsors added an o:line for themselves on the server they sponsored and whilst we do not believe that this was done with any malicious intent, more through thoughtlessness/negligence and having forgotten the expectations set out on our “Hosting a Server” page we feel that we are unable to comfortably and confidently continue the relationship.

Our number one priority has to be our target communities, the Free and Open Source Software communities that have chosen to make use of freenode in their internet activities.

Whilst we do not believe and have no evidence to indicate that any user traffic or data has been compromised, we would of course encourage you to change your passwords if you feel that this would make you more comfortable in continuing to use our services.

We can only apologise for this happening and we’d like to assure you that trust is incredibly important to us and that we are incredibly embarassed that this situation arose in the first place.

As a result of this we have just replaced our SSL certificates, so if you notice that these have changed then this is the reason why.

We will of course take this opportunity to remind all our sponsors of our expectations when it comes to providing services to freenode and our target communities.

Again, we apologise for any inconvenience and we hope that any loss of trust in the network that may have resulted from this incidence can be restored and that your projects will continue to feel comfortable using the network in future.

 

 

Over 9000 * 10

freenode has been growing slowly and steadily, breaking the next practically-useless-but-still-kinda-neat barrier of more than 90,000 concurrent connections at the same time. It’s very nice, and humbling, to know that we are able to enable so many people to communicate with each other.

I shouldn’t have added a month of leeway at the last second to my last prediction so the scary scary 100,000 is officially targeted for May 2014. Yes, the pace at which freenode is growing seems to be increasing ever so slightly.

Historic posts for those of you keeping track:

http://blog.freenode.net/2007/08/freenode-has-reached-40-000-users/

http://blog.freenode.net/2008/09/50000-active-users/

http://blog.freenode.net/2009/12/happy-new-year-2010/

http://blog.freenode.net/2011/01/freenode-70k/

http://blog.freenode.net/2012/04/80k/

Services upgrade and database prune

Hi
The long-awaited upgrade of services which we blogged about a while ago is now planned for this coming weekend, the 16th/17th June.

We anticipate up to an hour of services outage for this upgrade and prune to take place. We will notify the connected users closer to the time through the use of WALLOPS and/or globals, but please do plan ahead accordingly for a period of services unavailability.

We will be moving to Atheme 7, so, amongst other improvements, this will see the introduction of certificate-based authentication to services.

To use certificate based authentication, you would need firstly to generate a certificate, then add the certificate to your client, then tell nickserv about your certificate fingerprint. We’ll explain more about this in a future blog entry or on the freenode website in the near future.

Database prune

Every couple of years, freenode likes to get out the shears and prune the services database. Recently we broke the 80,000 usercount barrier, but the services stats are way ahead:

Sat 13:35:46 -OperServ(OperServ@services.)- Registered accounts: 446777
Sat 13:35:46 -OperServ(OperServ@services.)- Registered nicknames: 557497
Sat 13:35:47 -OperServ(OperServ@services.)- Registered channels: 141373

We’ve noticed that nearly half of the accounts shown there haven’t been used in the past 6 months! More importantly, over the past few months many people have noticed significant waits when issuing certain services commands – and we’d like to fix that.

Hopefully, the services upgrade should help with this, but we’re going to coincide this with a database prune.

As of the services upgrade date, any nicks unused for > 150 days are at risk of being dropped. This includes grouped nicks. The easy way to avoid this happening is to use each of your grouped nicks (while identified to the appropriate account) within the next few weeks – and to drop those that you don’t need anymore!

The testnet (testnet.freenode.net, port 9002. 9003 for SSL) is running a database snapshot from mid-March and will be periodically updated from the production network. This database instance is being regularly pruned – so check there to see how your account will be affected (use /msg nickserv info on both the production and test networks to see the differences).

Remember that testnet isn’t running a real-time duplicate of the production network, so when you use nicks which would be expired on the production network, they will still appear expired on testnet until the next database snapshot is migrated. Don’t worry though – the actual pruning will only occur on the current database at the time of upgrade.

On which note.. an upgrade date hasn’t been formally fixed but we’re aiming for mid-May.

Thanks, and don’t forget to test the testnet!

Help us test our services upgrade!

Very soon we will be upgrading your favourite network helpers… (no not erry…): NickServ, ChanServ, Alis etc. They’re currently connected to our testnet and we need your help with testing, looking for any issues which may affect the production network.

You can connect to our testnet at testnet.freenode.net port 9002 (or 9003 for SSL)

The full changelog is rather long and not all of the features offered by atheme are loaded on freenode. So to help you out, we’ve pulled out the highlights which we think deserve attention:

  • NickServ’s certfp module. (see /msg nickserv help cert and this link.)
  • NickServ will now notify you in real time of failed logins.
  • NickServ’s previous limit on password lengths has been increased.
  • ChanServ will still hand over single-# channels to freenode-staff on expiration of the channel founders, but the method has changed.
  • NickServ & ChanServ’s ‘set’ commands have had a general reorganisation behind the scenes. Nothing should be visibly different but it won’t hurt to check them!

Please note that the services database on the testnet is probably more than a few days old. Don’t be surprised if recent changes you have made on the production network aren’t replicated there.

We’re all in in #freenode on the testnet so please come find us there if you have any questions or bugs.

Finally, look out for a followup blogpost (hopefully quite soon) with some important information on the upgrade itself and our planned database cleanup!

Thanks for using freenode!

P.s. a full list of changes from atheme ~5.1 to ~6 can be found here

80k…

As of today, freenode had 81443 concurrent connections. Last Friday, the count lingered around 79987, presumably to make things more interesting for those of us keeping close track.

We passed this milestone one working day later than I predicted in January 2011, more than a year ago. Scary precision, I know. Extrapolation based on graphs and nothing else is a big no-no in the scientific circles, yet we really do grow at an exceptionally steady and predictable pace.

Still remembering the excitement when irc.openprojects.net surpassed 2,000 users, I have always had a knack for those numbers. Meaningless in and as of themselves, they do prove that we are able to provide a valuable service to our users and that more and more people apparently agree. freenode is not without growing pains, but I like to think that we are on a good path.

Once again, all of us would like to thank all our users and all our volunteers. It’s a fun ride for us and, hopefully, for you, too!

To make tracking our old posts regarding this easier, here goes:

http://blog.freenode.net/2007/08/freenode-has-reached-40-000-users/

http://blog.freenode.net/2008/09/50000-active-users/

http://blog.freenode.net/2009/12/happy-new-year-2010/

http://blog.freenode.net/2011/01/freenode-70k/

See you around July 2013 when we expect to hit 90k.. :)

Merry Christmas

It was the day before Christmas, when all across the network not a creature was stirring, not even a troll…

Another year is coming to a close, a year of FOSS, a year of collaboration, a year of getting to know exciting projects and contributors, a year plentiful in conferences and events. A year in which community was at the heart of everything freenode and the PDPC did.

I am sat here in a candle lit room that smells of Christmas spice whilst sipping eggnog and stealing a moment to myself to reflect… when I first started using freenode, the network had around 1200 users — some years on we’ve passed 76,000. It is great to see so many projects make use of the service! The PDPC is doing some exciting stuff, FOSSCON was arranged for the second year running in 2011, FOSSEVENTS is still going strong and Geeknics are being held across the globe. Sometimes it is a bit daunting, everyone involved with freenode (and other PDPC projects) volunteer their time and skills to help the communities — and whilst this is great, the volunteer roster isn’t growing at a rate matching the increase in users. We will be doing another “Call for volunteers” in the New Year and we’re looking to find some awesome people to add to the team.

On a more personal basis, I’ve had the pleasure of participating in some brilliant “real life” events this year – from the always brilliant OGGCAMP, arranged by the amazing people behind the Linux Outlaws and Ubuntu UK podcasts. OGGCAMP is the sort of event which attracts “my sort of geek” — people with which I really enjoy spending time and socialising and I am already looking forward to next years! I’ve also enjoyed a number of Ubuntu-UK events, from the recent release party to smaller happy hours to the rather interesting Christmas meal at Dans Le Noir, where we ate a surprise menu (consisting of glow in the dark scallops, ostrich and blue shark to name some items!) in complete darkness before wandering off to enjoy a few pints of ale. Now, this is where freenode communities are great — I am an avid fan of the Ubuntu project, however, I have never used the distribution (though as a Debian user I am sure I’d get on with it just fine.. right?) but through freenode I have, over the years, come to know a lot of Ubuntu contributors and users, and over time discovered that these are people I really enjoy spending my time and people I am proud to call my friends! And I love their social events, they are a lot of fun! So thank you for letting me be a part of your community despite not really being “one of you” ;)

On the subject of freenode, communities and Christmas parties — as freenode volunteers are scattered across the globe, few of us meet on a regular basis but we tend to try make an effort to get together for dinner and drinks at events such as fosdem. This year, we decided to have a Christmas party, surprisingly we ended up with 21 people attending, most of which had never met each other in person before. Our volunteers flew over from America and Europe and we all had a smashing weekend filled with good food, nice drink and much laughter. Thank you all for coming!

In 2012 I look forward to learning about more projects, old and new. I look forward to attending yet more conferences and meeting more of you in person! I look forward to another year together with our volunteers and our sponsors, and of course our users.

On which note I shall wrap this entry up, thank you for using freenode and wish you all a Merry Christmas!

 

 

ircd upgrades

We’ve got some ircd upgrades in the works!

You may remember several weeks ago that we upgraded our ircd on the production network. Since then, we’ve wanted to fine-tune some changes and make sure that the upgrade is more consistent with the old version.

Over the next few weeks, we’ll be looking to perform upgrades on the production network again. This will mean every server will reboot. A programme for the upgrades can be found at the end of this post (updated 13th Nov 2011).

In the meantime, please continue to help us to test the ircd at testnet.freenode.net port 9002 or 9003 for SSL (if you don’t get onto the first server that the DNS roundrobin gives you, keep trying!). Look for anything broken, inconsistent with previous versions (especially in terms of information release) or illogical. If serious issues are reported, we’ll aim to fix before upgrading, rather than having a further later upgrade. Please report issues to #freenode-seven on the production network.

Thanks!

NB: this list does not include servers invisible to users (eg hubs).
Week 1: Sun 13th Nov
-!- kornbluth.freenode.net Frankfurt, Germany
-!- zelazny.freenode.net Corvallis, OR, USA
-!- stross.freenode.net Corvallis, OR, USA (webchat backup)

Week 2: Sun 20th Nov
-!- barjavel.freenode.net Paris, FR
-!- wolfe.freenode.net Manchester, England
-!- hubbard.freenode.net Pittsburgh, PA, US

Week 3: Sun 27th Nov
-!- adams.freenode.net Budapest, HU, EU
-!- holmes.freenode.net London, UK
-!- sendak.freenode.net Vilnius, Lithuania, EU
-!- rowling.freenode.net Corvallis, OR, USA (webchat)

Week 4: Sun 4th Dec
-!- pratchett.freenode.net Rennes, France
-!- calvino.freenode.net Milan, IT
-!- leguin.freenode.net Ume?, SE, EU
-!- niven.freenode.net Corvallis, OR, USA

Week 5: Sun 11th Dec
-!- hitchcock.freenode.net Sofia, BG, EU
-!- gibson.freenode.net Oslo, Norway
-!- card.freenode.net Washington, DC, USA
-!- asimov.freenode.net TX, USA
-!- verne.freenode.net Newark, NJ, US

Unscheduled.
-!- roddenberry.freenode.net
-!- bartol.freenode.net
-!- brown.freenode.net
-!- anthony.freenode.net


Update: all upgrades are now complete.

New GRF-freenode process

As you might know, GRFs (Group Registration Forms) exist to form a relationship between a project and the PDPC (Peer Directed Projects Center). This relationship is relatively formal – personal details (address/tel no./etc) need to be shared by the project. For this reason, a severe backlog of GRFs has built up, since only a few staff have access to them (to protect this personal data). PDPC is the UK-based not-for-profit company which runs freenode. For most groups in our request backlog, their reason for registering is not to work with PDPC, but to gain a channel or cloak namespace on freenode. We’ve decided that running a separate, freenode-centric groups request system may help to move the system along. By requesting fewer details, we can open up this system to more staff, and hopefully keep on top of the queue of requests.

From now on, using a new, shorter form, projects can choose to file a GRF-f (for GRF-freenode) and submit a GRF for processing by freenode, rather than by PDPC. This sends details (no personal details, other than email address, will be required) to a system to which many more staff will have access. This new form will allow you to gain control of a channel and the right to issue cloaks much more quickly than previously, as we will double/treble the number of staff able to deal with requests. For now, please only apply if you are a ‘priority’ group – ie, you do not own the main channel of your namespace.

If you already have a group registered and approved with an old-style GRF, you do not need to do anything. Your registration remains valid. If you need to make changes to the registration, please contact staff on freenode who will, if appropriate, direct you to use the old (GRF) system. The GRF-f system cannot be used to update groups which filed under the GRF system.

If you have a request pending in the old GRF queue, you are welcome to re-file under the GRF-f system. This is likely to mean that your request will be dealt with much more quickly than otherwise. This approach supersedes the grfprocess@ system introduced a while back – unfortunately, we just weren’t able to keep up with requests to that address.

You might be wondering where all of this fits into the GMS (Groups Management System) masterplan. When GMS is ready, we may need to ask all projects registered under the GRF-f system, and likely some projects which are already registered, to re-file. The GMS system will allow us to dispense with GRF-fs, and just build project<=>PDPC relationships, since forms will be able to be processed much more quickly. To be clear – it is quite possible that any registration made now may be revoked if a registration is not re-filed after GMS is released. If this does become reality, as much warning as possible will be given.

We hope that this will change will counter some of the ill-feeling around the GRFs system. In effect, the mentality is shifting from one of “GMS will clear the GRFs backlog” to “GMS will help us to serve groups better”. We’re no longer waiting for GMS to clear the queue. We’re still looking for help with GMS: if you have Perl/Catalyst or web design experience and think you can help, join #freenode-gms.

Update 2012-06-09 – All group registration has been suspended whilst we evaluate the system and its implementation. A replacement should be available in due course, but for now it is not possible to register groups, and the link to the grf-f form has been removed.