Improved Mollom module with Webform support

We've just published the new 6.x-1.13 release for the Mollom module for Drupal. Coming about a month and a half after our last release, this release tweaks, extends, and improves the module code, and makes some important bug fixes.

First, the blacklisting UI has been enhanced to make it a much cleaner and more intuitive user experience and to allow for more granular blacklisting. To do so, we had to extend the underlying blacklist API with a new context parameter.

Work on Webform module integration also continued. To support users that have thousands of Webforms (apparently such users exist), we had to make some scalability improvements to the Mollom module's integration API. We're pleased to share that, thanks to these changes, Mollom integration has been committed to the Webform module and that Mollom support is part of the 6.x-3.0-beta4 release of Webform module.

A number of important bug fixes are present in this release as well, including several that involve audio CAPTCHAs and the upgrade path not working correctly. Upgrading to the new Mollom 6.x-1.13 release is highly recommended, especially if you experienced problems the last weeks. The Mollom module's unit tests have been improved and extended -- we now have more than 1400 assertions for the Mollom module alone!

Last but not least, we released a first alpha release of the Mollom module for Drupal 7. All of our automated tests pass, so we believe the module is in really good shape. In addition, the Drupal 7 alpha module was installed on Drupal Gardens sites so that should result in more real-life feedback too.

Thanks Daniel Kudwien (sun), Nathan Haug (quicksketch), Dave Reid, and others from the Mollom issue queue for their contributions to this latest release.

Mollom CAPTCHAs are "intelligent"

Every other week or so, someone asks me the following question: How are Mollom CAPTCHAs better than those created by CAPTCHA module?. This is an important question, and understanding it is central to understanding our philosophy with Mollom.

First, when using Mollom in "text analysis" mode, a CAPTCHA is only displayed when Mollom is uncertain about whether a message could be spam. Mollom analyzes the text of comments and combines that analysis with what it knows about the internal reputation of the posters, to determine whether a message is "spammy". Non-spam submissions are accepted without a CAPTCHA, and posts that are certainly spam are rejected automatically. By only presenting a CAPTCHA when necessary, we avoid penalizing normal (non-spamming) users with CAPTCHA challenges. The CAPTCHA module is different in that it does not perform text analysis and therefore must always display a CAPTCHA challenge.

Second, the Mollom module for Drupal has a "CAPTCHA only" mode, which is useful when clients would prefer not to use text analysis, or for when the forms have almost no text to analyze (like Drupal's user registration form). In "CAPTCHA only" mode, the user experience of the Mollom module is very similar to that of the CAPTCHA module -- the user is always prompted to complete a CAPTCHA in order to perform a certain operation. The similarity ends here, however. While the user experience is the same, the actual CAPTCHA generation is not. Mollom CAPTCHAs are "intelligent", in the sense that Mollom tracks the behavior and reputation of IP addresses from all sites using Mollom. A known spammer, operating from a known IP with a poor reputation, won't be able to complete a Mollom CAPTCHA no matter how hard he tries. And, as more users install Mollom, its performance increases as it learns from the additional data. A stand-alone module like CAPTCHA doesn't learn from user behavior, as it simply generates CAPTCHAs without regard to their context and delivery.

This second difference between the Mollom and other CAPTCHA modules is, in fact, huge. When we analyze our server logs, we see that 20% of all correctly completed CAPTCHAs are submitted by known spammers. Spammers don't seem to solve CAPTCHAs algorithmically; instead, they persuade humans to solve CAPTCHAs for them by using botnet infected machines. Two blog posts that detail this process are How to defeat Koobface and Breaking Koobface's CAPTCHA solving process. As spammers evolve and their arsenal of tools become increasingly powerful, CAPTCHA solutions must keep up to remain effective. We believe Mollom's "intelligent CAPTCHA" processing represents a significant benefit from traditional CAPTCHA generation and is one way we'll continue to stay a step ahead in our goal to eliminate posting spam.

Spammers set up shop at IBM.com

In this post we show one way spammers operate, using organizations like IBM to spam others. We show how spammers abuse an IBM wiki to create an online pharmacy via IBM.com. Once a page like this is in place, the spammers try to insert links to it in blog comments and community websites across the web to drive traffic to it, and to improve their search engine ranking.

While IBM is not using Mollom (maybe they should?), many of the targeted blogs and community websites are, which is how we discovered this problem. Watch the video below for more details. This problem is not specific to IBM. We found hundreds of similar spam pages on Google Sites, Yahoo, Amazon.com, Wordpress.com, Ning, Tripod.com, Xanga.com and more.

I recorded this video last week, and notified IBM about the existence of these spam pages several days ago.

New features for Drupal's Mollom module

It's been a year since our last significant changes to the Mollom module for Drupal, so we're excited to announce a major update. The most exciting new features in the new Mollom 1.11 release include:

  • Enables use of Mollom for any form; with Webform module integration almost completed
  • Includes an embedded audio CAPTCHA player that works on all browsers
  • Adds optional blacklist functionality to block spammers by text patterns and URLs
  • Adds an optional link to Mollom's privacy policy from forms protected via textual analysis
  • Adds support for serving CAPTCHAs over SSL (for Mollom Plus and Mollom Premium customers only)
  • Makes Mollom compatible with Pressflow
  • Integrates with CCK to change the position of the CAPTCHA field
  • Provides many more unit tests for continuous integration testing
  • Improves the APIs to enable module developers to better integrate with the Mollom module
  • Implements various usability improvements (e.g., better permission names and better error messages)
  • Updates several of the translations

We spent several months working on this release, and it marks our biggest upgrade to date. Upgrading requires some database updates, so remember to run update.php.

Thanks to Daniel F. Kudwien (sun), Dave Reid and Keith Smith for their contributions to the 1.11 release.

Mollom blacklisting and language detection APIs

If you're a regular reader of this blog, you know that Mollom is a continual work in progress. By studying how people use Mollom, by listening to feature requests, and by examining the plugins that our software partners and others have made available, we've introduced new ways to interact with Mollom.

First, we're announcing support for blacklisting. We introduced two new methods: one based on detecting the presence of user-specified URLs, and another that detects specific phrases or keywords. In both cases, Mollom maintains custom, site-specific URL and text blacklists, and knows to search for the presence of these links or phrases when analyzing text for your site. We're adding support for this API to the next version of the Mollom module for Drupal.

Second, we've implemented a new method that detects the language of any given text. We currently support detection of about 75 languages and this new functionality allows our end-users to take action based on posting language. It could be used to help segment web postings into different forums by language, or to help moderate the languages spoken on your site, for instance. The language detection API is used by some of our customers, but probably won't make it into the next version of the Mollom module for Drupal.

We've got other new features that we're working on as well, and will introduce them as they're ready. In the meantime, I'm excited to see what our plugin developers do with this new level of control.