websphinx
Class Link

java.lang.Object
  |
  +--websphinx.Region
        |
        +--websphinx.Element
              |
              +--websphinx.Link
All Implemented Interfaces:
Prioritized
Direct Known Subclasses:
Form, FormButton

public class Link
extends Element
implements Prioritized

Link to a Web page.

See Also:
Page

Field Summary
static int GET
          Use the HTTP GET method to download this link.
static int POST
          Use the HTTP POST method to access this link.
protected  java.net.URL url
           
 
Fields inherited from class websphinx.Element
child, endTag, parent, sibling, startTag
 
Fields inherited from class websphinx.Region
end, names, source, start, TRUE
 
Constructor Summary
Link(java.io.File file)
          Make a Link from a File.
Link(java.lang.String href)
          Make a Link from a string URL.
Link(Tag startTag, Tag endTag, java.net.URL base)
          Make a Link from a start tag and end tag and a base URL (for relative references).
Link(java.net.URL url)
          Make a Link from a URL.
 
Method Summary
 void discardContent()
          Eliminate all references to page content.
 void disconnect()
          Disconnect this link from its downloaded page (throwing away the page).
static java.net.URL FileToURL(java.io.File file)
          Convert a local filename to a URL.
 int getDepth()
          Get depth of link in crawl.
 java.lang.String getDirectory()
          Get the directory part of the link, like "/home/dir/".
 java.net.URL getDirectoryURL()
          Get the URL of a page's directory.
static java.net.URL getDirectoryURL(java.net.URL url)
          Get the URL of a page's directory.
 DownloadParameters getDownloadParameters()
          Get the download parameters used for this link.
 java.lang.String getFile()
          Get the filename part of the link, which includes the pathname and query but not the anchor reference.
 java.lang.String getFilename()
          Get the filename part of the link, like "index.html".
 java.lang.String getHost()
          Get the hostname of the link, like "www.cs.cmu.edu".
 int getMethod()
          Get the method used to access this link.
 Page getPage()
          Get the downloaded page to which the link points.
 java.net.URL getPageURL()
          Get the URL of a page, omitting any anchor reference (like #ref).
static java.net.URL getPageURL(java.net.URL url)
          Get the URL of a page, omitting any anchor reference (like #ref).
 java.net.URL getParentURL()
          Get the URL of a page's parent directory.
static java.net.URL getParentURL(java.net.URL url)
          Get the URL of a page's parent directory.
 int getPort()
          Get the port number of the link.
 float getPriority()
          Get the priority of the link in the crawl.
 java.lang.String getProtocol()
          Get the network protocol of the link, like "ftp" or "http".
 java.lang.String getQuery()
          Get the query part of the link.
 java.lang.String getRef()
          Get the anchor reference of the link, like "#ref".
 java.net.URL getServiceURL()
          Get the URL of a Web service, omitting any query or anchor reference.
static java.net.URL getServiceURL(java.net.URL url)
          Get the URL of a Web service, omitting any query or anchor reference.
 int getStatus()
          Get the status of the link.
 java.net.URL getURL()
          Get the URL.
static java.lang.String relativeTo(java.net.URL here, java.lang.String there)
           
static java.lang.String relativeTo(java.net.URL here, java.net.URL there)
           
 Tag replaceHref(java.lang.String newHref)
          Copy the link's start tag, replacing the URL.
 void setDownloadParameters(DownloadParameters dp)
          Set the download parameters used for this link.
 void setPage(Page page)
          Set the page corresponding to this link.
 void setPriority(float priority)
          Set the priority of the link in the crawl.
 void setStatus(int event)
          Set the status of the link.
 void setText(java.lang.String text)
          Set the tagless-text representation of this region.
 java.lang.String toDescription()
          Generate a human-readable description of the link.
 java.lang.String toText()
          Convert the region to tagless text.
 java.lang.String toURL()
          Convert the link's URL to a String
static java.lang.String toURLDelimiters(java.lang.String path)
           
protected  java.net.URL urlFromHref(Tag tag, java.net.URL base)
          Construct the URL for a link element, from its start tag and a base URL (for relative references).
static java.io.File URLToFile(java.net.URL url)
          Convert a file: URL to a filename appropriate to the current system platform.
 
Methods inherited from class websphinx.Element
enumerateHTMLAttributes, getChild, getEndTag, getHTMLAttribute, getHTMLAttribute, getNext, getParent, getSibling, getStartTag, getTagName, hasHTMLAttribute
 
Methods inherited from class websphinx.Region
enumerateObjectLabels, findEnd, findStart, getEnd, getField, getFields, getLabel, getLabel, getLength, getNumericLabel, getObjectLabel, getObjectLabels, getRootElement, getSource, getStart, hasAllLabels, hasAllLabels, hasAnyLabels, hasAnyLabels, hasLabel, removeLabel, setField, setFields, setLabel, setLabel, setObjectLabel, span, toHTML, toString, toTags
 
Methods inherited from class java.lang.Object
clone, equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait
 

Field Detail

url

protected java.net.URL url

GET

public static final int GET
Use the HTTP GET method to download this link.


POST

public static final int POST
Use the HTTP POST method to access this link.

Constructor Detail

Link

public Link(Tag startTag,
            Tag endTag,
            java.net.URL base)
     throws java.net.MalformedURLException
Make a Link from a start tag and end tag and a base URL (for relative references). The tags must be on the same page.

Parameters:
startTag - Start tag of element
endTag - End tag of element
base - Base URL used for relative references

Link

public Link(java.net.URL url)
Make a Link from a URL.


Link

public Link(java.io.File file)
     throws java.net.MalformedURLException
Make a Link from a File.


Link

public Link(java.lang.String href)
     throws java.net.MalformedURLException
Make a Link from a string URL.

Throws:
java.net.MalformedURLException - if the URL is invalid
Method Detail

discardContent

public void discardContent()
Eliminate all references to page content.


disconnect

public void disconnect()
Disconnect this link from its downloaded page (throwing away the page).


getDepth

public int getDepth()
Get depth of link in crawl.

Returns:
depth of link from root (depth of roots is 0)

getURL

public java.net.URL getURL()
Get the URL.

Returns:
the URL of the link

getProtocol

public java.lang.String getProtocol()
Get the network protocol of the link, like "ftp" or "http".

Returns:
the protocol portion of the link's URL

getHost

public java.lang.String getHost()
Get the hostname of the link, like "www.cs.cmu.edu".

Returns:
the hostname portion of the link's URL

getPort

public int getPort()
Get the port number of the link.

Returns:
the port number of the link's URL, or -1 if no port number is explicitly specified in the URL

getFile

public java.lang.String getFile()
Get the filename part of the link, which includes the pathname and query but not the anchor reference. Equivalent to getURL().getFile().

Returns:
the filename portion of the link's URL

getDirectory

public java.lang.String getDirectory()
Get the directory part of the link, like "/home/dir/". Always starts and ends with '/'.

Returns:
the directory portion of the link's URL

getFilename

public java.lang.String getFilename()
Get the filename part of the link, like "index.html". Never contains '/'; may be the empty string.

Returns:
the filename portion of the link's URL

getQuery

public java.lang.String getQuery()
Get the query part of the link. Either starts with a '?', or is empty.

Returns:
the query portion of the link's URL

getRef

public java.lang.String getRef()
Get the anchor reference of the link, like "#ref". Either starts with '#', or is empty.

Returns:
the anchor reference portion of the link's URL

getPageURL

public java.net.URL getPageURL()
Get the URL of a page, omitting any anchor reference (like #ref).

Returns:
the URL sans anchor reference

getPageURL

public static java.net.URL getPageURL(java.net.URL url)
Get the URL of a page, omitting any anchor reference (like #ref).

Returns:
the URL sans anchor reference

getServiceURL

public java.net.URL getServiceURL()
Get the URL of a Web service, omitting any query or anchor reference.

Returns:
the URL sans query and anchor reference

getServiceURL

public static java.net.URL getServiceURL(java.net.URL url)
Get the URL of a Web service, omitting any query or anchor reference.

Returns:
the URL sans query and anchor reference

getDirectoryURL

public java.net.URL getDirectoryURL()
Get the URL of a page's directory.

Returns:
the URL sans filename, query and anchor reference

getDirectoryURL

public static java.net.URL getDirectoryURL(java.net.URL url)
Get the URL of a page's directory.

Returns:
the URL sans filename, query and anchor reference

getParentURL

public java.net.URL getParentURL()
Get the URL of a page's parent directory.

Returns:
the URL sans filename, query and anchor reference

getParentURL

public static java.net.URL getParentURL(java.net.URL url)
Get the URL of a page's parent directory.

Returns:
the URL sans filename, query and anchor reference

relativeTo

public static java.lang.String relativeTo(java.net.URL here,
                                          java.net.URL there)

relativeTo

public static java.lang.String relativeTo(java.net.URL here,
                                          java.lang.String there)

FileToURL

public static java.net.URL FileToURL(java.io.File file)
                              throws java.net.MalformedURLException
Convert a local filename to a URL. For example, if the filename is "C:\FOO\BAR\BAZ", the resulting URL is "file:/C:/FOO/BAR/BAZ".

Parameters:
file - File to convert
Returns:
URL corresponding to file

URLToFile

public static java.io.File URLToFile(java.net.URL url)
                              throws java.net.MalformedURLException
Convert a file: URL to a filename appropriate to the current system platform. For example, on MS Windows, if the URL is "file:/FOO/BAR/BAZ", the resulting filename is "\FOO\BAR\BAZ".

Parameters:
url - URL to convert
Returns:
File corresponding to url
Throws:
java.net.MalformedURLException - if url is not a file: URL.

toURLDelimiters

public static java.lang.String toURLDelimiters(java.lang.String path)

getPage

public Page getPage()
Get the downloaded page to which the link points.

Returns:
the Page object, or null if the page hasn't been downloaded.

setPage

public void setPage(Page page)
Set the page corresponding to this link.

Parameters:
page - Page to which this link points

getMethod

public int getMethod()
Get the method used to access this link.

Returns:
GET or POST.

toURL

public java.lang.String toURL()
Convert the link's URL to a String

Returns:
the URL represented as a string

toDescription

public java.lang.String toDescription()
Generate a human-readable description of the link.

Returns:
a description of the link, in the form "[url]".

toText

public java.lang.String toText()
Convert the region to tagless text.

Overrides:
toText in class Region
Returns:
a string consisting of the text in the page contained by this region

setText

public void setText(java.lang.String text)
Set the tagless-text representation of this region.

Parameters:
text - a string consisting of the text in the page contained by this region

urlFromHref

protected java.net.URL urlFromHref(Tag tag,
                                   java.net.URL base)
                            throws java.net.MalformedURLException
Construct the URL for a link element, from its start tag and a base URL (for relative references).

Parameters:
tag - Start tag of link, such as <A HREF="/foo/index.html">.
base - Base URL used for relative references
Returns:
URL to which the link points

replaceHref

public Tag replaceHref(java.lang.String newHref)
Copy the link's start tag, replacing the URL. Note that the name of the attribute containing the URL varies from tag to tag: sometimes it is called HREF, sometimes SRC, sometimes CODE, etc. This method changes the appropriate attribute for this tag.

Parameters:
newHref - New URL or relative reference; e.g. "http://www.cs.cmu.edu/" or "/foo/index.html".
Returns:
copy of this link's start tag with its URL attribute replaced. The copy is a region of a fresh page containing only the tag.

getStatus

public int getStatus()
Get the status of the link. Possible values are defined in LinkEvent.

Returns:
last event that happened to this link

setStatus

public void setStatus(int event)
Set the status of the link. Possible values are defined in LinkEvent.

Parameters:
event - the event that just happened to this link

getPriority

public float getPriority()
Get the priority of the link in the crawl.

Specified by:
getPriority in interface Prioritized

setPriority

public void setPriority(float priority)
Set the priority of the link in the crawl.


getDownloadParameters

public DownloadParameters getDownloadParameters()
Get the download parameters used for this link. Default is null.


setDownloadParameters

public void setDownloadParameters(DownloadParameters dp)
Set the download parameters used for this link.