Collecting data from open sources: why is it risky when information is scraped from other people's databases?

Website as a Database

Information systematized on websites (including catalogs, social networks, and marketplaces) is often an officially created database. The rights to this database may belong to the company that developed the corresponding website. Accordingly, using these numerous listed materials for commercial purposes without the permission of their rights holder is considered a violation.

In 2017, hearings began in Russian courts in the case of V Kontakte LLC against Double Data LLC.

"Double Data" launched several programs:

• A search engine (allows users to find a person's required account and provide a link to it);

• Software for visualizing information associated with links;

• Software that detects "duplicate" user accounts.

V Kontakte LLC argued that the aforementioned software:

• operates without proper authorization;

• is intended for scoring purposes;

• violates their rights to the established database.

This was followed by a court filing.

The case was heard over several years, proceeding through a series of court instances in turn.

Ultimately, the Court of Intellectual Rights approved a settlement agreement between the two corporations.

Legal analysts, reviewing the rulings of the Court of Intellectual Rights (CIR) (even though no final position was adopted on key issues), note the following important details:

1. Even if users themselves uploaded information to the website's database, this does not mean that the site's creator (organizer) holds no rights to this database.

2. The manufacturer of a database is considered to be the entity that invested substantial resources into creating the product.

How to draw the line between "processing is permitted" and "processing is prohibited"?

First, freedom of access to information is enshrined in the Constitution. Individual facts (including product prices, their specifications, publicly available materials, and similar information) do not fall into the category of copyrighted works.

Secondly, the database creator obtains exclusive rights to it if financial, organizational, or other costs were invested in the creation of the product.

Thirdly, an entity cannot systematically use a substantial part of another party's database without the permission of the rights holder. This means, for example, that extracting individual elements or an insignificant portion of the product is permitted.

The line between substantial and insignificant use in this context is very thin. It may be drawn, for instance, based on extraction volume: if you use non-critical data from a maximum of one or two social media profiles, this may be considered legal; if you extract data from a thousand profiles on a commercial basis, then analyze it and share it with third parties, this may be recognized as an unlawful action.

Fourthly, the structure of the created database falls under legal protection.

Basic rules for automated information scraping (parsing)

Do not parse:

• Content related to personal data (unless permission is explicitly stipulated by law);

• Published exclusive content.

When parsing:

• Check what the website owner specifies in the robots.txt file (it defines which sections of the site are allowed for search engine indexing and which are prohibited);

• Do not send frequent requests to the system (they may be classified as an attack).

And there were no parsing-related cases in the West?

There were.

One of the most significant (and interesting) cases is the confrontation between LinkedIn social network and hiQ Labs.

LinkedIn representatives stated that their opponents (competitors) are collecting publicly available user data. To stop this, the social network blocked access to public profiles.

In court, the situation unfolded as follows:

• In the first instance, LinkedIn was prohibited from restricting hiQ Labs' access to data;

• In the appeal, this decision was upheld (since users themselves made their profiles public, there is no grounds to restrict them in such an expression of their will).

The case reached the Supreme Court, which remanded the case for review.

Motivating its decision, the Supreme Court cited the case of law enforcement officer Robert Van Buren, who, abusing his official position, accessed the police database and, for a bribe, provided another person with information about a female driver’s license plate number. The court found that in this case, illegal unauthorized access to a protected computer had occurred. That is: the police officer had the authority to obtain information, but used it for an unlawful purpose.

Taking a similar ruling into account, the appellate court found that the concept of unauthorized access cannot be applied to websites hosted in the public domain. It did not restrict access to publicly available data, and did not find a violation of the relevant law.

But companies that operate such websites have not left them unprotected: restrictions can be stipulated in the user agreement, and non-compliance with these restrictions will result in a lawsuit.