org.erowid.sperowider.simple
Class SimpleSpiderModel

java.lang.Object
  org.erowid.sperowider.simple.SimpleSpiderModel

All Implemented Interfaces:

ISimpleSpiderModel

public class SimpleSpiderModel

extends Object

implements ISimpleSpiderModel

Interface to our data storage. The basic process is : 1. A found URL is proposed to the data store 2. If that found URL has already been proposed, it is discarded 3. If it has neither been proposed nor redirected to, it is added to the candidate list. OR 1. A new URL is requested from the data store 2. If that URL causes a redirect, the redirected-to URL is proposed to the data store 3. If that URL returns data, it is parsed

Version:

$Id: SimpleSpiderModel.java,v 1.6 2005/01/16 05:03:01 gurustu Exp $

Author:

sstatman@real.com

Constructor Summary

SimpleSpiderModel()
Default constructor

Method Summary

void	addFoundURL(String foundIn, String found) Proposes a URL for future parsing.
void	destroy() Does nothing.
List	getFoundURLs(String sourceURL) This method is not supported in this model.
Collection	getInvalidURLs() Returns the set of invalid URLs
List	getSourceURLs(String foundURL) This method is not supported in this model.
int	getSpiderQueueSize() The number of URLs left in the queue.
String	getUnspideredUrl() Returns a URL for parsing.
boolean	isSpiderMapSupported() Returns false always, as getSourceURLs(String) and getFoundURLs(String) are not supported in this model.
void	markInvalidURL(String givenURL, int responseCode, String message) Marks a URL as bad

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

SimpleSpiderModel

public SimpleSpiderModel()

Default constructor

Method Detail

addFoundURL

public void addFoundURL(String foundIn,
                        String found)

Proposes a URL for future parsing.

Specified by:

addFoundURL in interface ISimpleSpiderModel

getUnspideredUrl

public String getUnspideredUrl()

Returns a URL for parsing. If that URL redirects, it should be discarded.

Specified by:

getUnspideredUrl in interface ISimpleSpiderModel

getSpiderQueueSize

public int getSpiderQueueSize()

The number of URLs left in the queue.

Specified by:

getSpiderQueueSize in interface ISimpleSpiderModel

destroy

public void destroy()

Does nothing.

Specified by:

destroy in interface ISimpleSpiderModel

getFoundURLs

public List getFoundURLs(String sourceURL)
                  throws UnsupportedOperationException

This method is not supported in this model.

Specified by:

getFoundURLs in interface ISimpleSpiderModel

Throws:

UnsupportedOperationException - If the model does not support this method

getSourceURLs

public List getSourceURLs(String foundURL)
                   throws UnsupportedOperationException

This method is not supported in this model.

Specified by:

getSourceURLs in interface ISimpleSpiderModel

Throws:

UnsupportedOperationException - If the model does not support this method

markInvalidURL

public void markInvalidURL(String givenURL,
                           int responseCode,
                           String message)

Marks a URL as bad

Specified by:

markInvalidURL in interface ISimpleSpiderModel

getInvalidURLs

public Collection getInvalidURLs()

Returns the set of invalid URLs

Specified by:

getInvalidURLs in interface ISimpleSpiderModel

isSpiderMapSupported

public boolean isSpiderMapSupported()

Returns false always, as getSourceURLs(String) and getFoundURLs(String) are not supported in this model.

Specified by:

isSpiderMapSupported in interface ISimpleSpiderModel