Spiders, bots and other creepy crawlers: protecting your company website
By Gina Durham
Websites contain a wealth of information. Unfortunately, the human internet users you hope are accessing your site are not the only ones attracted to it. Automated tools, frequently referred to as spiders, bots and screen scrapers, may be crawling your company website too.
Given the potential of the internet to consolidate and manipulate information, automated data aggregation has become a business model for many companies. Using specialised software applications, a scraper can almost instantaneously access numerous websites, submit queries to the sites, extract data from them and gather copies of the actual web pages viewed. Websites containing unique collections of proprietary, pricing and consumer information are frequent targets of scrapers.
Some aggregators, such as reputable online travel sites that provide a collection of airline and hotel options, may ‘scrape’ source websites with the authorisation of the website owners. Others, however, attack competitors’ sites, engaging in unauthorised scraping of large volumes of data. Yet others may crawl sites searching for email addresses, credit card numbers or other sensitive information…
If you are registered and logged in to the site, click on the link below to read the rest of the DLA Piper briefing. If not, please register or sign in with your details below.
Sign in or Register to continue reading this article
It's quick, easy and free!
It takes just 5 minutes to register. Answer a few simple questions and once completed you’ll have instant access.Register now
Why register to The Lawyer
In-depth, expert analysis into the stories behind the headlines from our leading team of journalists.
Identify the major players and business opportunities within a particular region through our series of free, special reports.
Receive your pick of The Lawyer's daily and weekly email newsletters, tailored by practice area, region and job function.
More relevant to you
To continue providing the best analysis, insight and news across the legal market we are collecting some information about who you are, what you do and where you work to improve The Lawyer and make it more relevant to you.
News from DLA Piper
News from The Lawyer
Briefings from DLA Piper
The Australian Taxation Office released a draft ruling on the Goods and Services Tax treatment of bitcoin transactions on 20 August 2014.
DLA Piper’s ‘Life sciences: patent extension strategies and antitrust global update’ video covers global antitrust and competition issues including product hopping and reverse payment patents.
Analysis from The Lawyer
Cross-border work and political tensions are dominating this year’s entries for Finance Team of the Year at The Lawyer Awards.
Regulators are ramping up the pressure in the aftermath of recession, leaving firms to compete for compliance and restructuring work