Netcraft

From Computer Laboratory Group Design Projects
Jump to navigationJump to search

Andy Ide <asi@netcraft.com>

Suggested alternative - not proceeding Cluster Mail

Original suggestion:

Whilst spam filters catch some malicious emails, attackers' content still makes it into users' inboxes. These emails include malware which steals user data; ransomware which requires payment to decrypt user data; and spear-phishing attacks which impersonate people and organisations to gain access to victims' money and details. Studies show that phishing alone cost UK consumers £137 million per year, so it is vital that users are better assisted and educated to deal with the threat confidently and accurately. Your task is to write a plugin for web or desktop email clients to identify these threats and protect users from harm, whilst maximising harm to the attacker. It should be able to rate and explain the threat to users, allow them to report malicious emails to services which can disable the threat and disrupt the attacker, and teach the user to spot future malicious emails.

There are many ways you could detect suspicious traits in an email. SPF, DKIM, DMARC and similar standards are a great step forward for proving authenticity of email from-addresses but contemporary mail clients do not present this information in a concise and helpful manner to their user communities. Making use of this information is very helpful in defending against threats delivered as SMTP forgeries, but other mails where the headers are not forged may also be malicious.

Consider making use of other suspicious signs in headers, or try applying fuzzy hashing and Bayesian filters to address these - we can provide a large sample of training email data if this is useful. The more malicious mails the tool can report, the more it protects users, so consider making use of the spam folder as well. You could provide a "phish myself" training mode where the plugin inserts fake phishing emails into user inboxes to help develop their competence.

Feedback

Thanks for this. At present, my concerns were that the brief might not be sufficiently novel (from a user perspective, it's a variation on a familiar product category), that it might require too much integration with existing commodity software such as mail clients and desktop file managers, and that spam/phishing detection is a well-defined specialist field in which undergraduates are unlikely to make any real technical advance.

It would be possible to narrow down the scope, to focus on only one of these aspects, as in the previous project examples I sent. But then there may be an issue with dividing up the remaining technical work among a team of six.


Previous suggestions:

The contact, who will act as the client, is Robert Duncan (rad@netcraft.com)

Phishing is a pervasive form of identity theft affecting just about every Internet service that involves authentication. The UK Cards Association recently estimated that phishing attacks increased by around 80% between 2010 & 2011, and that UK online bank fraud losses were in the region of £35 million.

The Netcraft anti-phishing community is effectively a giant neighbourhood watch scheme, empowering the most alert and most expert members to defend everyone within the community against phishing attacks. Once the first recipients of a phishing mail have reported the target URL, it is blocked for community members if they subsequently access the that URL. Netcraft's feed is used in all the major web browsers and it is also licensed by many of the leading anti-virus, content filtering, web-hosting and domain registration companies

Netcraft's system owes its success to the ingenuity of its community of phishing site reporters. Although many members are small scale reporters forwarding individual phishing attacks received in their own mail, some have clearly invested effort in automation to promptly find and report phishing sites.

One technique that has been successful is to register a domain, create mail addresses within the domain, and an internet presence for those mail addresses which is likely to attract the attention of spammers, fraudsters and their programs, then filter the mail received to report plausible phishing attacks, and use machine learning to tune these filters according to which reports are accepted. Community members have also developed approaches to phish disseminated through social media sites and search engines.

The project is to write a piece of software that can, by whatever means, contribute effectively to the community, finding and reporting as many phishing sites as possible, as quickly as possible, whilst keeping false positive incorrect reports to a respectably small level.

Netcraft will provide accept/reject mails on reports, a leader board such that the project can assess their progress and various incentives when milestones are reached.