next | previous | forward | backward | up | top | index | toc | home

getWWW -- get a web page

Description

getWWW URL -- obtain the contents of the web page, together with the http headers, at the address given by URL, from an http server.
getWWW(URL,TEXT) -- obtain the contents of the web page addressed by URL from an http server, using the POST method, provided with TEXT.

This doesn't work under Solaris because Sun doesn't provide sockets or name service to statically linked programs like this one.

Accessing a secure web site (whose URL begins with https:) depends on your having installed openssl on your system.

In this example we show how to use regex to separate the header lines in the response from the content.

i1 : p = getWWW "http://www.math.uiuc.edu/Macaulay2/Makefile"

o1 = HTTP/1.1 200 OK
     Date: Sat, 15 Jul 2006 04:45:04 GMT
     Server: Apache/2.0.50 (Fedora)
     Last-Modified: Tue, 16 May 2006 16:46:53 GMT
     ETag: "401df-1c0-8cd6b940"
     Accept-Ranges: bytes
     Content-Length: 448
     Connection: close
     Content-Type: text/plain; charset=UTF-8

     all :
             $(MAKE) -C $(HOME)/local/htdig M2

     VERSION = 0.9.8
     DIR = $(HOME)/local.Linux/encap/Macaulay2-$(VERSION)/
     EXC =   --exclude="*.m2" \
             --exclude=".\#*" \
             --exclude=".linkdir" \
             --exclude="*.out"
     PATHS = share/Macaulay2 share/doc/Macaulay2
     TARDIR = doc-$(VERSION)

     listdoc :
             tar chf - -C $(DIR) $(EXC) $(PATHS) | tar tfv - -C $(TARDIR)
     copydoc : $(TARDIR)
             tar chf - -C $(DIR) $(EXC) $(PATHS) | tar xf - -C $(TARDIR)
     $(TARDIR) :; mkdir "$@"
i2 : r = regex("\r\n\r\n",p)

o2 = {(261, 4)}

o2 : List
i3 : substring(p,r#0#0+4)

o3 = all :
             $(MAKE) -C $(HOME)/local/htdig M2

     VERSION = 0.9.8
     DIR = $(HOME)/local.Linux/encap/Macaulay2-$(VERSION)/
     EXC =   --exclude="*.m2" \
             --exclude=".\#*" \
             --exclude=".linkdir" \
             --exclude="*.out"
     PATHS = share/Macaulay2 share/doc/Macaulay2
     TARDIR = doc-$(VERSION)

     listdoc :
             tar chf - -C $(DIR) $(EXC) $(PATHS) | tar tfv - -C $(TARDIR)
     copydoc : $(TARDIR)
             tar chf - -C $(DIR) $(EXC) $(PATHS) | tar xf - -C $(TARDIR)
     $(TARDIR) :; mkdir "$@"
i4 : peek oo

o4 = "all :\n\t$(MAKE) -C $(HOME)/local/htdig M2\n\nVERSION = 0.9.8\nDIR =
     $(HOME)/local.Linux/encap/Macaulay2-$(VERSION)/\nEXC =
     --exclude=\"*.m2\" \\\n\t--exclude=\".\\#*\"
     \\\n\t--exclude=\".linkdir\" \\\n\t--exclude=\"*.out\"\nPATHS =
     share/Macaulay2 share/doc/Macaulay2\nTARDIR = doc-$(VERSION)\n\nlistdoc
     :\n\ttar chf - -C $(DIR) $(EXC) $(PATHS) | tar tfv - -C
     $(TARDIR)\ncopydoc : $(TARDIR)\n\ttar chf - -C $(DIR) $(EXC) $(PATHS) |
     tar xf - -C $(TARDIR)\n$(TARDIR) :; mkdir \"$@\"\n"
i5 : substring(p,0,r#0#0+2)

o5 = HTTP/1.1 200 OK
     Date: Sat, 15 Jul 2006 04:45:04 GMT
     Server: Apache/2.0.50 (Fedora)
     Last-Modified: Tue, 16 May 2006 16:46:53 GMT
     ETag: "401df-1c0-8cd6b940"
     Accept-Ranges: bytes
     Content-Length: 448
     Connection: close
     Content-Type: text/plain; charset=UTF-8
i6 : peek oo

o6 = "HTTP/1.1 200 OK\r\nDate: Sat, 15 Jul 2006 04:45:04 GMT\r\nServer:
     Apache/2.0.50 (Fedora)\r\nLast-Modified: Tue, 16 May 2006 16:46:53
     GMT\r\nETag: \"401df-1c0-8cd6b940\"\r\nAccept-Ranges:
     bytes\r\nContent-Length: 448\r\nConnection: close\r\nContent-Type:
     text/plain; charset=UTF-8\r\n"

Ways to use getWWW :