Networking for Cowards - Part 3 - HTTP POST queries

Previous topic - Next topic

Kitty Hello

Howdy folks! In the last tutorials we learned about how to interact between GLBasic programs, and GLBasic programs and a web server we wrote on our own.
This time we talk to a web server we did NOT write on our own.

Sometimes you see a webpage that offers any kind of service for you. Like weather or the current TV program. But when you click on the button, the URL does yield any of the information you queried. So, how does the webserver know what you selected in the html page?

If you look at the source code of such a website, you will see a form code like this:
Code (glbasic) Select

<form method="POST" action="whatever.php">

And this is the key to that door. The browser sends the request not with HTTP GET (where you see whatever.php?name=John&lastname=Doe), but in a HTTP POST request header that is hidden from the browser user. Using this technique, the browser is also able to upload larger files to the server, since the HTTP GET is limited to a few hundred characters length.

A simple HTTP request looks like this:
Code (glbasic) Select

   POST /somepage.php HTTP/1.0
   Content-Type: application/form-data
   Content-Length: 0

Be sure to end each line with "\r\n" and end the block with "\r\n" again.

Now, each of the <input ...> fields in the <form> tag in the html page is a variable that you must send to the server. "name" is the name of that property and "value" the value.

In order to separate the inputs and have multiline inputs, you should use a boundary string, that tells the http server when a new variable block begins.

Code (glbasic) Select

Content-Type: multipart/form-data, boundary=--------------XXXX1234--

Then send each variable's content like this:
Code (glbasic) Select

Content-Disposition: form-data; name="username"

John Doe

Pay attention! The boundary block starts with "--" and then the boundary string! So 2 more dashes at the beginning. Again, after each boundary header, add a 2nd "\r\n" to indicate the start of the value data. This way you can even send binary files:

Code (glbasic) Select

Content-Disposition: form-data; name="thefile"; filename="C:\foo.png"
Content-Transfer-Encoding: binary
Content-Type: image/png
Content-Type: application/octet-stream


The "image/png" just is additional information for the webserver, that could dissallow certain types for uploading. Play nice and send him was he wants.

Finally, put an emtpy boundary line at the bottom.

Make sure that the "Content-Length: " in the topmost header is set to the length of all data in bytes.

Then open a connection to that server:
Code (glbasic) Select

sock% = SOCK_TCPCONNECT(server$, port%)

and send him the whole header.
Code (glbasic) Select

IF SOCK_TCPSEND(sock%, send$) > 0

if it was successful, the server is starting to write you some bytes in return. What you now get is a HTTP header plus the file information the webbrowser usually displays. The header and the contents are separated by a double "\r\n" sequence.

Here's a complete example that uploads a png file to a webserver:
Code (glbasic) Select

// --------------------------------- //
// Project: HttpPost
// Start: Thursday, March 05, 2009
// IDE Version: 6.174

// make a png image
// -----------------------------------------
DRAWRECT 0,0,240,80,RGB(0,0,200)
PRINT "Hello World", 0,0

GRABSPRITE 0,0,0,240,80
SAVESPRITE "httppostimg.png", 0
// -----------------------------------------

LOCAL boundary$ // a boundary string that delimits the header parts
LOCAL postdata$ // what you post

// make a boundary string. That's a few "--" and some random characters
boundary$ = "----------------bOunDarY-------"+INTEGER(GETTIMERALL()*1000.0)+"-"+RND(99999)+"--"

// post variables
INC postdata$, AddHttpPostVar$("check_password", YOUR PASSWORD HERE, boundary$)
INC postdata$, AddHttpPostVar$("submit", "true", boundary$)
// also upload an image -> You can adjust this function to upload whatever you want, easily.
INC postdata$, AddHttpPostPngImage$("file[0]", "httppostimg.png", boundary$)
// ...

// Now get the website
// -----------------------------------------
// host it on a http website, using HTTP POST technique
LOCAL server$="" YOUR SERVER HERE // the server URL
LOCAL curl$  ="/pix/upload.php" YOUR PATH HERE // the path on the server (start with / )
LOCAL proxy$ =""  // leave this empty for a direct connection
LOCAL port%=3128                // use port 80 for direct connection
LOCAL website$
website$ = HttpPost$(proxy$, server$, curl$, port%, boundary$, postdata$)
// -----------------------------------------

// find the image url
// -----------------------------------------
LOCAL img_url$
LOCAL pos% = INSTR(website$, "Direct Link: <a href=", 0)
IF pos>0
pos% = INSTR(website$, "\"", pos)
IF pos>0
LOCAL endp%
endp% = INSTR(website$, "\"", pos+1)
IF endp>0
img_url$ = MID$(website$, pos+1, endp-pos-1)
// -----------------------------------------

// make a preview HTML page
// -----------------------------------------
OPENFILE(1, "img.html", FALSE)
WRITELINE 1, "<html><head></head><body><img src=\""+img_url$+"\"></body></html>"
// -----------------------------------------

// ---------------------------------------------------------------------
// Send a HTTP POST Request and return what the webbrowser would display
// ---------------------------------------------------------------------
FUNCTION HttpPost$: proxy$, server$, curl$, port%, boundary$, postdata$
LOCAL header$ // the HTTP post header

// Fix proxy urls
IF LEN(proxy$)
curl$ = "http://"+server$+curl$
server$ = proxy$

// end post data
postdata$ = postdata$ + "--"+boundary$+"--"

LOCAL send$
send$ = "POST "+curl$+" HTTP/1.0\r\n"
INC send$, "User-Agent: GLBasic\r\n"
INC send$, "Host:"+server$+"\r\n"
INC send$, "Accept: */*\r\n"
INC send$, "Content-Type: multipart/form-data, boundary="+boundary$+"\r\n"
INC send$, "Content-Length: "+LEN(postdata$) + "\r\n"
INC send$, "\r\n"
INC send$, postdata$

LOCAL sock%, file$
sock% = SOCK_TCPCONNECT(server$, port%)
IF SOCK_TCPSEND(sock%, send$) > 0
LOCAL chunk$, rv%
rv% = SOCK_RECV(sock%, chunk$, 1024)
CASE -2 // non blocking socket not would block
BREAK // ok, reading is done
INC file$, chunk$
// jippieh! We got a response

// split HTTP header and actual file information apart
LOCAL pos%
pos = INSTR(file$, "\r\n\r\n")
IF pos>=0
header$ = MID$(file$, 0, pos%)
file$   = MID$(file$, pos%+4)

RETURN file$

// ---------------------------------------------------------------------
// add a variable to a HTTP POST query
// ---------------------------------------------------------------------
FUNCTION AddHttpPostVar$: varname$, value$, boundary$
LOCAL postdata$
postdata$ = "--"+boundary$+"\r\nContent-Disposition: form-data; name=\""+varname$+"\"\r\n\r\n"+value$+"\r\n"
RETURN postdata$

// ---------------------------------------------------------------------
// upload a png image variable to a HTTP POST query
// ---------------------------------------------------------------------
FUNCTION AddHttpPostPngImage$: varname$, filepath$, boundary$
LOCAL postdata$
LOCAL content$

IF OPENFILE(1, filepath$, TRUE)
// chr$(0) is a string of length "1", that contains a '\0' character!
content$ = content$ + CHR$(b%)

postdata$ = "--"+boundary$+"\r\n"
INC postdata$, "Content-Disposition: form-data; name=\""+varname$+"\"; filename=\""+filepath$+"\"\r\n"
INC postdata$, "Content-Transfer-Encoding: binary\r\n"
INC postdata$, "Content-Type: image/png\r\n"
INC postdata$, "Content-Type: application/octet-stream\r\n\r\n" + content$ + "\r\n";

RETURN postdata$

Beware! You need GLBasic update >= 6.184, since I fixed a bug that lets you have CHR$(0) characters in a string now.

The PHP code I use for that image uploader is

Text was too  long, I'll attach the file.

The program above will write a img.html file, that displays the uploaded image.
If you put such a ".httaccess" file in your "pix" directory, you can view all the images as I did:
Code (glbasic) Select

AddType application/x-httpd-php5       .php  
AddType application/x-httpd-php4       .php4    
AddType application/x-httpd-php5       .php5

mod_gzip_on Yes

Options +Indexes

IndexIgnore *.php *.htaccess tn*.*

See it in action:

[attachment deleted by admin]




"It is practically impossible to teach good programming style to students that have had prior exposure to BASIC.  As potential programmers, they are mentally mutilated beyond hope of regeneration."
(E. W. Dijkstra)