the DMCA protects search engine page caching, indexing, etc.? Not so fast.

		Print
		Email weblog link
		Discuss
		Blog this

Sid Steward
Nov. 07, 2005 02:03 PM
Permalink

URL: http://www.arl.org/info/frn/copy/band.html...

* Update *

When I first posted this, I felt confident that the cited laws protected web crawling by search engines. After receiving critical comments (shown below) I looked more closely. I now believe my critic was correct. These laws, ushed in as part of the DMCA, don't directly address web crawling. I have made some changes to this post to suit.

When folks talk about Google Library and fair use, Google's current practice of indexing web pages is commonly offered as a sort of precedent. 'Google can index web pages, so it should also be able to index books.' Or even: 'if Google can't index books, then it can't index the web.' However, the DMCA gives online service providers special leeway to handle online information. Does the DMCA protect web crawling? Maybe. Maybe not.

I'm not a lawyer, and I'm not suggesting that the DMCA is wonderful. I just wanted to follow up on some of the things I had read. This is what I found. Further digging would surely reveal more, but this satisfies my curiousity for now.

I'm copying this from the Association of Research Libraries web site.

Excerpted from THE DIGITAL MILLENNIUM COPYRIGHT ACT by Jonathan Band, Morrision & Foerster LLP, copyright 2001 Association of Research Libraries.

TITLE II: ONLINE SERVICE PROVIDER LIABILITY

Title II of the Act limits an online service provider's ("OSP") liability for copyright infringement in several important situations. Because the term "service provider" is defined extremely broadly in some instances - "a provider of online services or network access, or the operator of facilities therefor" ³³ - many entities which are not in the business of providing online services may nonetheless take advantage of Title II's protection. ³⁴

The exemptions from liability that the DMCA creates are additional to any defense that an OSP might have under copyright law or any other law.³⁵ In essence, the Act creates certain "safe harbors" for specified OSP activity. If an activity falls within the safe harbor, then the OSP qualifies for the exemption from liability; if the activity does not come within the safe harbor, then the questions of whether the activity in fact constitutes infringement and whether the OSP has any defense are to be decided under traditional copyright analysis. In addition, whether an OSP qualifies for any particular exemption is determined independently of whether the OSP qualifies for any other exemption. ³⁶

General Conditions for Eligibility

Termination Policy. To be eligible for any of the exemptions, an OSP must adopt, reasonably implement, and inform its subscribers and account holders (its "Users") of, a policy providing for termination of Users who are repeat infringers.³⁷

Accommodation of Technical Measures. In addition, an OSP must accommodate and not interfere with "standard" technical measures used by copyright owners to identify and protect copyrighted works. Such technical measures might include, for example, digital watermarks or technological means for preventing copying of a work. In order to qualify as "standard", such a measure must have been developed by a broad consensus of copyright owners and OSPs in a fair multi-industry process, must be available to anyone on reasonable and nondiscriminatory terms, and must not impose substantial costs on OSPs or substantial burdens on OSP systems.³⁸

No Need to Monitor or Access. The Act makes clear that in order to qualify for the exemptions, an OSP does not need to monitor its service or affirmatively seek out information about copyright infringement on its service (except as part of the standard technical measures discussed in the previous paragraph). In addition, the Act states that an OSP does not have to access, remove, or block material in order to qualify for its exemptions if such action is prohibited by law (such as, for example, the Electronic Communications Privacy Act). ³⁹

Safe Harbors for System Storage and Information Locating Tools

The most straightforward exemptions in the Act cover two common OSP activities: (1) storing material (such as a Web page or chat room, for example) on an OSP's system at the request of a User and (2) referring Users to material at other online locations by means of, for example, a search engine, a list of recommended sites, or a hypertext link.⁴⁰

The Act limits an OSPs liability for copyright infringement based on the material stored or referred to if the OSP meets certain conditions:

(1) the OSP doesn't actually know that the material is infringing;

(2) the OSP isn't aware of information from which the infringing nature of the material is apparent;

(3) if the OSP acquires such knowledge or awareness, the OSP acts expeditiously to remove or block access to the material;

(4) the OSP doesn't get a financial benefit directly attributable to the infringing material (for example, a special fee paid by each party that accesses the material) while having the right and ability to control the material; and

(5) the OSP complies with the "notice and take down" provisions of the Act (discussed below).⁴¹

Safe Harbors for System Caching

A third safe harbor in the Act limits an OSP's liability for system caching, in which an OSP makes a temporary copy of popular Internet material requested by a User so that the OSP can deliver that copy to subsequent Users, which can be done more quickly and efficiently than obtaining the original material for each subsequent User. ⁴²

This exemption applies to material (a) that is originally placed online by someone other than the OSP (the "Originator") and (b) that is transmitted from the Originator, through the OSP's system, to a third party at that third party's request. To qualify for the exemption from liability for the intermediate and temporary storage of such material, the OSP must meet the following conditions:

(1) the OSP's storage of the cached material must be made through an automatic technical process and must be for the purpose of providing the material to subsequent Users who request the material;

(2) the OSP must transmit the cached material to subsequent Users without modifying its content;

(3) the OSP must comply with any rules on updating the cached material that are specified by the Originator using a generally accepted industry standard protocol, as long as such rules are not used by the Originator to prevent or unreasonably impair system caching;

(4) the OSP must not interfere with technology associated with the cached material that returns certain information to the Originator, as long as such technology doesn't significantly interfere with the performance of the OSP's system and is consistent with generally accepted industry standard protocols;

(5) if the Originator has placed conditions (such as payment of a fee or entry of a password) on access to the cached material, the OSP must allow access to the cached material only to subsequent Users that have met such conditions; and

(6) if the original material from which the cached copy was made has been removed or blocked and a copyright owner provides notice to the OSP (pursuant to certain "notice and take-down" provisions discussed below), the OSP must act expeditiously to remove or block access to the cached material that the copyright owner alleges is infringing.⁴³

Safe Harbors for Transmission and Routing

A final safe harbor in the Act covers an OSP's transmission, routing, or providing connections for material through the OSP's system and for intermediate and transient storage of material in the course of such activity. In essence, this safe harbor covers an OSP's activities in acting as a conduit for material travelling between other parties.

To qualify for this exemption, several conditions must be met:

(1) the transmission of the material must have been initiated or directed by someone other than the OSP;

(2) the activities covered by the exemption must be carried out through an automatic technical process and not by any selection of material by the OSP;

(3) the OSP must not select the recipients of the material except as an automatic response to another person's request;

(4) the OSP must not make any copy of the material ordinarily accessible to anyone other than intended recipients and must not keep any copy for longer than reasonably necessary for the OSP's transmission, routing, or connection; and

(5) the OSP must not modify the content of the material as it transmits it through its system.⁴⁴

Extent of Exemptions from Liability

The safe harbors of the Act provide somewhat different limitations on different types of remedies usually available for copyright infringement.

Monetary Relief. If an OSP's activity qualifies for any of the safe harbors in the Act, then the OSP is not liable for any monetary relief for claims of copyright infringement based on that activity. Monetary relief includes damages, court costs, attorney's fees, and any other form of monetary payment.

Injunctions. If an OSP qualifies for a safe harbor under the Act, then the possible injunctive relief against the OSP is limited. Under any safe harbor, a court may issue an injunction restraining an OSP from providing access to an identified User engaging in infringement by terminating the User's specified accounts. With respect to the safe harbors for system caching, system storage, and information location tools, a court can also issue an injunction restraining an OSP from providing access to infringing material residing at a particular online site on the OSP's system. Any other injunctive relief must be necessary to prevent infringement of specified material at a particular online location and must be the least burdensome to the OSP among comparably effective forms of relief. With respect to transmission and routing, a court can also issue an injunction ordering an OSP to take specific reasonable steps to block access to an identified online location outside the U.S. ⁴⁵

The Act also sets forth several additional considerations, including the burden on an OSP's system, the technical feasibility, and the interference with noninfringing material, that a court must consider in the case of all the safe harbors in deciding whether to grant injunctive relief. The DMCA further limits the liability of nonprofit institutions of higher education that act as OSPs for the infringing acts of their faculty and graduate students when performing teaching or research functions.

"Permission is granted to reproduce and distribute copies of the work for nonprofit educational or library purposes, provided that copies are distributed at or below cost, and that the author, source, and copyright notice are included on each copy."

Sid Steward is a programmer, writer and entrepreneur. He maintains the PDF Toolkit and wrote PDF Hacks.

Comment on this weblog
You must be logged in to the O'Reilly Network to post a comment.

Showing messages 1 through 6 of 6.

• example of Google Print FUD
  2005-11-10 09:01:14  Sid Steward | [Reply | View]
  
  
  Here's an example of the misinformation I'm talking about, published by Lawrence Lessig. I'll simply quote my email to the author.
  
  Hello-
  
  Just read your Wired piece Google's Tough Call:
  http://lookleap.com/wired.com/a4
  
  You say:
  
  "Indeed, their claims about Google represent the biggest landgrab in the history of the Internet, and if taken seriously, will chill a wide range of innovation. Because if the AAP is right, it's not Google Print that's illegal. The outlaw is Google itself - and Yahoo!, and MSN Search, and the Internet Archive, and every other technology that makes knowledge useful in a digital age."
  
  "Think about Google's core business: It copies whatever content it finds on the Web and puts that content in an index. It doesn't ask the copyright owner first, though it does exclude content if asked. Thus, Google wants to do for books exactly what it has always done for the Web. Why should one be illegal and the other different?"
  
  Which suggests that indexing the web is protected only by fair use. That if Google loses its fair use battle with print publishers, it will no longer be able to index the web.
  
  However, the DMCA includes sections specifically intended to allow Google, Yahoo, etc. to index and cache online information.
  
  Please see:
  the DMCA protects search engine page caching, indexing, etc.
  http://www.oreillynet.com/pub/wlg/8358
  
  And:
  The Digital Millennium Copyright Act
  http://www.arl.org/info/frn/copy/band.html
  
  You might also find this interesting:
  copyright law lets libraries distribute Twilight Zone material in full
  http://www.oreillynet.com/pub/wlg/8359
  
  Cheers-
  
  Sid
  
• example of Google Print FUD
  2005-11-10 16:38:07  Gordon Mohr | [Reply | View]
  
  
  As noted in my message yesterday, I believe you are misreading the relevant statute. Nothing in it allows Google to, on its own initiative, make copies of material it finds on the web. (The caching/transmission/storage clauses all provide only for immunity when the activity is triggered by others; the info-location-tool clause only applies to "referring or linking".)
  
  So why can Google retrieve those copies, index them, and in one form or another (the 'cache' or reverse-index) maintain 'copies' of the material? Fair use.
  
  - Gordon
  
• apology
  2005-11-10 21:11:16  Sid Steward | [Reply | View]
  
  
  I emailed Lawrence Lessig again with this apology:
  
  Hello-
  
  One of my readers made me scrutinize the DMCA more closely, so I realized the case wasn't as clear-cut as I thought. I apologize for my rash words.
  
  Cheers-
  
  Sid
  
• example of Google Print FUD
  2005-11-10 19:10:05  Sid Steward | [Reply | View]
  
  
  Thanks for the follow-up. Sorry for my late reply. Maybe you're right. Looking closely at the law, it doesn't offer clear-cut protection for services that crawl the web. But I think you could interpret the section on caching to apply to search engines.
  
  I didn't quote the text of the law, above, but rather a commentary. In hope of clarification, I followed the author's notes and looked up some of the material. Namely, section 512:
  http://lookleap.com/www4.law.cornell.edu/a1
  
  
  For example, the above commentary says about caching:
  
  This exemption applies to material (a) that is originally placed online by someone other than the OSP (the "Originator") and (b) that is transmitted from the Originator, through the OSP's system, to a third party at that third party's request.
  
  
  The related text of the law says:
  
  (1) Limitation on liability.— A service provider shall not be liable for monetary relief, or, except as provided in subsection (j), for injunctive or other equitable relief, for infringement of copyright by reason of the intermediate and temporary storage of material on a system or network controlled or operated by or for the service provider in a case in which—
  
  (A) the material is made available online by a person other than the service provider;
  
  (B) the material is transmitted from the person described in subparagraph (A) through the system or network to a person other than the person described in subparagraph (A) at the direction of that other person; and
  
  (C) the storage is carried out through an automatic technical process for the purpose of making the material available to users of the system or network who, after the material is transmitted as described in subparagraph (B), request access to the material from the person described in subparagraph (A),
  
  
  On first blush, both sound like the third party must request the material before it can be cached. I think that was the thrust of your argument. However, I believe they also apply in cases where the OSP caches the online content /before/ the third party requests it. In other words, the OSP could fetch the material from the publisher before the reader requests it. It would simply be part of its transmission.
  
  Any takers? ;-)
  
  
  However, the text of the law on caching prohibits altering the material (text not shown, here). So even if caching is okay, search engines would need another mechanism (such as fair use) to display excerpts of these pages to users.
  
  Sid

• Not so fast...
  2005-11-09 23:09:32  Gordon Mohr | [Reply | View]
  
  
  Google has great lawyers which may be able to find a way to make the safe harbors apply.
  
  But, I think you're misreading the safe harbors' applicability. In particular, the 'Caching', 'Transmission', and 'Information Residing' (Storage) safe harbors all require that the actual copying/transmitting/caching/storing happen "at the direction of" someone other than the OSP itself, or be "made available online" by someone else. For Google Print, it is arguably the OSP itself which is causing the transmission/caching/storage. (Though, perhaps they could argue their partner libraries are pushing the books into their index, giving them a safe harbor until notified that specific books must be removed.)
  
  Even the 'information location tool' safe harbor doesn't look applicable to me, because it specifically provides safe harbor "for infringement of copyright by reason of the provider referring or linking users to an online location containing infringing material or infringing activity." It provides no specific safe harbor for the act of creating a copy, merely for linking to truly infringing material.
  
  I think Google Print is legal based on other law and fair use precedent. First, every step of the process is independently legal: libraries *may* scan books for preservation and indexing purposes; sharing the index so constructed should not itself be considered a copy; listing books that satisfy search queries with a small amount of surrounding context fits the same fair use tests that allow web search results. And then, after all steps are followed, the end result does not involve any entity -- the libraries, Google, or users -- possessing any illegal copies of the copyrighted work [*]. So on neither the specific steps nor the net result has any 'illegal copying' occurred.
  
  - Gordon
  
  [*] UNLESS, the courts somehow decide that a factual word index to a copyrighted work is itself covered by the work's copyright... in which case, almost all search engine web indexing would be illegal too, because the indexes are (by-and-large) created without specific prior authorization.
• Not so fast...
  2005-11-10 21:04:13  Sid Steward | [Reply | View]
  
  
  At the end of the day, I ~would~ like to see book search succeed. In fact it's something I've thought about for years.

---

Showing messages 1 through 6 of 6.

Return to weblogs.oreilly.com.