When bots go bad..

First off, allow me to apologise to all users affected by the recent “client killing” rampage of our utility bot; Syn. She appears to have gotten into the Halloween spirit a bit too much!

You may have noticed a large number of people disconnecting from freenode with the reason ‘Nick collision from syn.’ We feel we should explain what happened.

For those of you not already familiar with her, syn is a utility bot that, amongst other duties, regulates gateway access to the network. This could be web gateways such as CGI:IRC or our own webchat, NAT gateways, or some conferences and shell services. One of the things that she does, for web gateways in particular, is to match the reported IP address (hex-encoded in the ident field) against network bans, and deny the connection if a match is found.

It was this particular part that had an unfortunate pair of bugs resulting in the incident you observed. Firstly, in using sscanf() to detect a hex-encoded IP address in the ident field, the validation was not quite strict enough — any ident that *began* with a series of valid hexadecimal characters (the digits 0-9 and letters a-f) would result in a number being decoded. In normal circumstances, this would be relatively harmless as the resulting IP is clearly invalid — in most cases, it would begin 0.0.0., and not match any network bans.

Unfortunately, there was a second bug introduced more recently as part of a performance fix. This meant that in certain cases, a K:line whose host part contained wildcards would incorrectly match against these invalid IP addresses.

Each of these, taken in isolation, would be relatively innocuous, and so they slipped under the radar and made it into production. The combination of the two, however, had rather disastrous results.

We apologise, and welcome you to castigate our developers and staff for our incompetence and for allowing these bugs to make it into production.

Wikipedia licensing change?

Wikipedia to change license to Creative Commons BY-SA?  — From http://creativecommons.org/weblog/entry/13967

A community vote is now underway, hopefully one of the final steps in the process the migration of Wikipedia (actually Wikipedias, as each language is its own site, and also other Wikimedia Foundation sites) to using Creative Commons Attribution-ShareAlike as its primary content license.

This migration would be a huge boost for the free culture movement, and for Wikipedia and Creative Commons — until the migration happens there is an unnecessary licensing barrier between the most important free culture project (Wikipedia of course, currently under the Free Documentation License, intended for software documentation) and most other free culture projects and individual creators, which use the aforementioned CC BY-SA license.

To qualify to vote, one must have made 25 edits to a Wikimedia site prior to March 15. Make sure you’re logged in to the project on which you qualify, and you should see a site notice at the top of each page that looks like the image below (red outline added around notice).

If you are eligible, please head over to http://creativecommons.org/weblog/entry/13967 and find out how to cast your vote!

Free-as-in-node episode 3 released!

The third installment of  free-as-in-node has just been released! Head on over to http://podcast.freenode.net to listen or download.

In this episode, you’ll hear all about freenodes april fools joke and our exploding growth (currently at around 57,000 online users).  Also discussed is the php testfest with input from mgdm (Michael Maclean) in this weeks pimp my project, and then a bit of soapboxing and the announcement of the new “freenode party” in UK politics, so that we too can partake of free-as-in-shoes-your-taxpayers-bought-you.  Make sure you sign our petition before Christel shuts down freenode!

In addition, we’ll let you know whats going on in the world with our take on the talks between Sun and IBM, Debian eating FreeBSD, the new Gnome 3.0 roadmap, and news on TomTom’s settlement with Microsoft.  We’ll also talk about Fedora11beta with updates on our experience on the next episode.

We also discuss the “geeknic” once again with the announcement of yet another geeknic, this one in the Philadelphia area in less than a week! If you can make it, bring us a canoe or kayak.

This week closes again with a wonderful morsel from Bryn_S.  We’ll see you in two weeks!

free-as-in-node: Episode 2 release

We’re happy to announce we’ve just released our second episode of the new freenode podcast, free-as-in-node!  In this episode, we talk about Gary’s recent offenses, geeknics and k-lining park-goers.  We also meet with Dave “Daviey” Walker from Ubuntu to discuss the Ubuntu COC and Ubuntu Locos during this episodes Pimp my project. We continue looking at the recent Microsoft and TomTom legal actions and share our views on them.

During the new freeform discussion segment of the show, we discuss facebook monitoring (and government monitoring in general) and more broadly all the counter terrorism actions being taken by government, including the bomb-resistant sandwich bag.

Head on over to http://podcast.freenode.net to listen in and to share your comments. See you there!

The Beauty of #freenode

#freenode channel, as it currently exists, is a veritable work of art – people who come to the channel are nearly always provided help or referred somewhere for the answer. The really beautiful thing about #freenode, though, is that at least 50% of the help provided comes from network users (rather than freenode staff).

Almost two years ago, the channel #freenode was recreated with new guidelines and philosophy consistent with freenode’s. The ideas weren’t that new – they had already been (somewhat) in use in the old #tapthru channel. The activity in #freenode is generally within the channel guidelines, which may be found at http://freenode.net/poundfreenode.shtml and is highly recommended for anyone planning to participate in the channel.

Whilst many help channels utilize specialized staff to answer questions, one of the most refreshing things about #freenode is that anyone who knows the answer to a question can, and often does, provide the answer or help to the user in need – regardless of status or staff-ness. Numerous users lurk in the channel, either to learn from others’ questions, help other users through their queries or generally just to take up space :)

Some of the more common questions can be, and are, answered by a multitude of people. Keep in mind that being on staff is not a prerequisite to having the correct answer to your question! Here are some common questions:

  • How do I register a nick? A: http://freenode.net/faq.shtml#nicksetup
  • How do I register a channel? A: http://freenode.net/policy.shtml#channelnaming /msg ChanServ help register http://freenode.net/group_registration.shtml
  • How do I set auto-ops on a channel? A: http://freenode.net/channel_guidelines.shtml
  • What is a cloak and how do I get one? A: First set your nick up this way (http://freenode.net/channel_guidelines.shtml), and then message a staff member. Some users whose accounts have been registered recently may be asked to wait a short time before being eligible for a cloak. Cloaks are privileges, not rights – they may be removed in the event of misconduct on the network.
  • There is someone trolling my channel! What should I do? A: freenode strongly encourages the idea of catalysing and has gone so far as to make it part of its policy for staff and official network channels, as well as encouraging others to use the same principles. For two helpful guides on freenode’s catalyst policy, please see http://freenode.net/catalysts.shtml and http://blog.freenode.net/2007/02/the-heart-and-mind-of-a-catalyst/. Part of dealing with trolling is understanding the motivations of the troll. Feel free to read this blog post on the subject or catch a staffer for more ideas: http://blog.freenode.net/2007/05/silence-is-golden-handling-trolls-and-spammers/
  • I’ve lost my password! Can someone help me regain access to my nick? A: If you set your nick up properly when you registered, staff is able to send you a password reset key. Ask in #freenode and when a staffer is available, he/she will be happy to do so.
  • I would like to use a nick that is already registered, will you drop it for me? A: again, staff can assist you. However, try running /msg NickServ info $nick – be sure it’s at least 60 days unused. Then /nick to the nick. Staff will not drop the nick unless you are using it when you ask for the drop. There are situations where, even if a nick is unused for at least 60 days, staff cannot or will not drop it. Be prepared to find another nick if that is the case.
  • I’ve been banned from a channel! Let me back in! A: #freenode is not the place to ask. If you have been banned from a channel, you need to contact the operators of the channel and request to be unbanned. May I suggest doing so politely? No matter how indignant you are, demanding to be unbanned is likely to not serve your goal. To find channel operators, try /msg ChanServ access #channame list
  • How do I find a specific channel if I don’t know the name? A: You can try using ALIS. /msg ALIS help

These are just a few of the many and varied requests in #freenode. Please feel free to feel free to hang out, learn, help and listen!

Canonical Pledges Matching Funds

Thanks to all of you for the donations you have contributed so far towards our fundraising drive. freenode has a few interesting fundraising events and ideas up its sleeves, so hopefully there will be a few more updates to the blog with more details in weeks to come! Today, freenode and the PDPC announces one of a number of exciting updates to the fundraising process:

Mark Shuttleworth and the Canonical team have kindly agreed to provide matching funding for up to £1700, which is enough to meet our £5000 goal). Effective immediately, every donation you make will have a matching contribution, thereby doubling in value! This is an extraordinarily kind gesture and we at freenode would like to thank Mark and Canonical, and the Ubuntu community, for directly and indirectly making it happen!

[Announcement] Network problems

Yesterday evening, freenode had network problems related to the recent spambot infestation. Despite the recent server patch, the underlying infestation continues and has created some temporary resource issues. Several servers restarts ensued and network staff rearranged the main server rotation to compensate for the problem.

The network is once again running smoothly. However, in order to resolve remaining resource issues, staff has scheduled server restarts at 6am UTC (10pm in San Francisco, 7am in Stockholm, 5pm in Sydney). The restarts will affect most of the network but should be very brief.

Other than the problems yesterday evening, the network has been running well, and staff considers the upgrade to be a qualified success. We’ll keep you posted in case of additional problems. Thanks much for your patience!

[Announcement] Temporary DNS Change

For the last 18 hours, freenode experienced a large-scale nuisancebot infestation. We temporarily removed the irc.freenode.net rotation and replaced it with chat.freenode.net, in order to try to stem the tide of spambots hitting the network. In the process of doing so, we discovered a new methodology for analyzing the spambot net and produced a patched server version to mitigate the effects of the infestation. As of a few minutes ago, all freenode servers have been upgraded to this new code.

We’re not certain how long the patch will work. Botnets are modified over time by their creators and this one will likely be no exception. We hope we can utilize the new methodology to handle subsequent generations of botnets, but there’s no guarantee our efforts will be successful. We believe that it will be helpful to make a gradual, permanent cutover of the main rotation hostname from irc.* to chat.* to make it somewhat less likely we’ll experience future problems.

If you’re a client developer or you run an IRC network list site, please do not list chat.freenode.net, in your client or site without first emailing staff at freenode dot net to make sure it’s okay to do so.

Finally, we want to thank you all for your patience and understanding. You’ve helped us stay on track and resolve the current difficulties, and we appreciate it. Apologies for the inconvenience, and thank you for using freenode!

[Announcement] DNS change

freenode is currently experiencing a large-scale nuisancebot infestation. We’ve temporarily removed the irc.freenode.net rotation, but you can find us on chat.freenode.net. If you’re a client developer or you run an IRC network list site, please do not list this new hostname without first emailing staff at freenode dot net to make sure it’s okay to do so. Apologies for the inconvenience, and thank you for using the network!