Networking for Cowards - Part 3 - HTTP POST queries

Previous topic - Next topic

Kitty Hello

Howdy folks! In the last tutorials we learned about how to interact between GLBasic programs, and GLBasic programs and a web server we wrote on our own.
This time we talk to a web server we did NOT write on our own.

Sometimes you see a webpage that offers any kind of service for you. Like weather or the current TV program. But when you click on the button, the URL does yield any of the information you queried. So, how does the webserver know what you selected in the html page?

If you look at the source code of such a website, you will see a form code like this:
Code (glbasic) Select

<form method="POST" action="whatever.php">
...
</form>


And this is the key to that door. The browser sends the request not with HTTP GET (where you see whatever.php?name=John&lastname=Doe), but in a HTTP POST request header that is hidden from the browser user. Using this technique, the browser is also able to upload larger files to the server, since the HTTP GET is limited to a few hundred characters length.

A simple HTTP request looks like this:
Code (glbasic) Select

   POST /somepage.php HTTP/1.0
   Host: example.com
   Content-Type: application/form-data
   Content-Length: 0



Be sure to end each line with "\r\n" and end the block with "\r\n" again.

Now, each of the <input ...> fields in the <form> tag in the html page is a variable that you must send to the server. "name" is the name of that property and "value" the value.

In order to separate the inputs and have multiline inputs, you should use a boundary string, that tells the http server when a new variable block begins.

Code (glbasic) Select

Content-Type: multipart/form-data, boundary=--------------XXXX1234--


Then send each variable's content like this:
Code (glbasic) Select

----------------XXXX1234--
Content-Disposition: form-data; name="username"

John Doe

Pay attention! The boundary block starts with "--" and then the boundary string! So 2 more dashes at the beginning. Again, after each boundary header, add a 2nd "\r\n" to indicate the start of the value data. This way you can even send binary files:

Code (glbasic) Select

----------------XXXX1234--
Content-Disposition: form-data; name="thefile"; filename="C:\foo.png"
Content-Transfer-Encoding: binary
Content-Type: image/png
Content-Type: application/octet-stream

BINARY-DATA_GOES_HERE


The "image/png" just is additional information for the webserver, that could dissallow certain types for uploading. Play nice and send him was he wants.

Finally, put an emtpy boundary line at the bottom.

Make sure that the "Content-Length: " in the topmost header is set to the length of all data in bytes.

Then open a connection to that server:
Code (glbasic) Select

SOCK_INIT()
sock% = SOCK_TCPCONNECT(server$, port%)

and send him the whole header.
Code (glbasic) Select

IF SOCK_TCPSEND(sock%, send$) > 0

if it was successful, the server is starting to write you some bytes in return. What you now get is a HTTP header plus the file information the webbrowser usually displays. The header and the contents are separated by a double "\r\n" sequence.

Here's a complete example that uploads a png file to a webserver:
Code (glbasic) Select

// --------------------------------- //
// Project: HttpPost
// Start: Thursday, March 05, 2009
// IDE Version: 6.174


// make a png image
// -----------------------------------------
DRAWRECT 0,0,240,80,RGB(0,0,200)
PRINT "Hello World", 0,0

GRABSPRITE 0,0,0,240,80
SAVESPRITE "httppostimg.png", 0
SHOWSCREEN
// -----------------------------------------



LOCAL boundary$ // a boundary string that delimits the header parts
LOCAL postdata$ // what you post

// make a boundary string. That's a few "--" and some random characters
boundary$ = "----------------bOunDarY-------"+INTEGER(GETTIMERALL()*1000.0)+"-"+RND(99999)+"--"

// post variables
INC postdata$, AddHttpPostVar$("check_password", YOUR PASSWORD HERE, boundary$)
INC postdata$, AddHttpPostVar$("submit", "true", boundary$)
// also upload an image -> You can adjust this function to upload whatever you want, easily.
INC postdata$, AddHttpPostPngImage$("file[0]", "httppostimg.png", boundary$)
// ...


// Now get the website
// -----------------------------------------
// host it on a http website, using HTTP POST technique
LOCAL server$="www.glbasic.com" YOUR SERVER HERE // the server URL
LOCAL curl$  ="/pix/upload.php" YOUR PATH HERE // the path on the server (start with / )
LOCAL proxy$ ="192.168.168.50"  // leave this empty for a direct connection
LOCAL port%=3128                // use port 80 for direct connection
LOCAL website$
website$ = HttpPost$(proxy$, server$, curl$, port%, boundary$, postdata$)
// -----------------------------------------


// find the image url
// -----------------------------------------
LOCAL img_url$
LOCAL pos% = INSTR(website$, "Direct Link: <a href=", 0)
IF pos>0
pos% = INSTR(website$, "\"", pos)
IF pos>0
LOCAL endp%
endp% = INSTR(website$, "\"", pos+1)
IF endp>0
img_url$ = MID$(website$, pos+1, endp-pos-1)
ENDIF
ENDIF
ENDIF
// -----------------------------------------

// make a preview HTML page
// -----------------------------------------
OPENFILE(1, "img.html", FALSE)
WRITELINE 1, "<html><head></head><body><img src=\""+img_url$+"\"></body></html>"
CLOSEFILE 1
// -----------------------------------------



// ---------------------------------------------------------------------
// Send a HTTP POST Request and return what the webbrowser would display
// ---------------------------------------------------------------------
FUNCTION HttpPost$: proxy$, server$, curl$, port%, boundary$, postdata$
LOCAL header$ // the HTTP post header

// Fix proxy urls
IF LEN(proxy$)
curl$ = "http://"+server$+curl$
server$ = proxy$
ENDIF

// end post data
postdata$ = postdata$ + "--"+boundary$+"--"

LOCAL send$
send$ = "POST "+curl$+" HTTP/1.0\r\n"
INC send$, "User-Agent: GLBasic\r\n"
INC send$, "Host:"+server$+"\r\n"
INC send$, "Accept: */*\r\n"
INC send$, "Content-Type: multipart/form-data, boundary="+boundary$+"\r\n"
INC send$, "Content-Length: "+LEN(postdata$) + "\r\n"
INC send$, "\r\n"
INC send$, postdata$

LOCAL sock%, file$
SOCK_INIT()
sock% = SOCK_TCPCONNECT(server$, port%)
IF SOCK_TCPSEND(sock%, send$) > 0
LOCAL chunk$, rv%
WHILE TRUE
chunk$=""
rv% = SOCK_RECV(sock%, chunk$, 1024)
SELECT rv%
CASE -1
DEBUG NETGETLASTERROR$()+"\n"
file$=""
BREAK
CASE -2 // non blocking socket not would block
SLEEP 5
CASE 0
BREAK // ok, reading is done
CASE >0
INC file$, chunk$
ENDSELECT
WEND
// jippieh! We got a response

// split HTTP header and actual file information apart
LOCAL pos%
pos = INSTR(file$, "\r\n\r\n")
IF pos>=0
header$ = MID$(file$, 0, pos%)
file$   = MID$(file$, pos%+4)
ELSE
header$=""
ENDIF
ENDIF
SOCK_CLOSE(sock%)

RETURN file$
ENDFUNCTION




// ---------------------------------------------------------------------
// add a variable to a HTTP POST query
// ---------------------------------------------------------------------
FUNCTION AddHttpPostVar$: varname$, value$, boundary$
LOCAL postdata$
postdata$ = "--"+boundary$+"\r\nContent-Disposition: form-data; name=\""+varname$+"\"\r\n\r\n"+value$+"\r\n"
RETURN postdata$
ENDFUNCTION

// ---------------------------------------------------------------------
// upload a png image variable to a HTTP POST query
// ---------------------------------------------------------------------
FUNCTION AddHttpPostPngImage$: varname$, filepath$, boundary$
LOCAL postdata$
LOCAL content$

IF OPENFILE(1, filepath$, TRUE)
LOCAL b%
WHILE ENDOFFILE(1)=FALSE
READBYTE 1, b%
// chr$(0) is a string of length "1", that contains a '\0' character!
content$ = content$ + CHR$(b%)
WEND
CLOSEFILE 1

postdata$ = "--"+boundary$+"\r\n"
INC postdata$, "Content-Disposition: form-data; name=\""+varname$+"\"; filename=\""+filepath$+"\"\r\n"
INC postdata$, "Content-Transfer-Encoding: binary\r\n"
INC postdata$, "Content-Type: image/png\r\n"
INC postdata$, "Content-Type: application/octet-stream\r\n\r\n" + content$ + "\r\n";
ENDIF

RETURN postdata$
ENDFUNCTION


Beware! You need GLBasic update >= 6.184, since I fixed a bug that lets you have CHR$(0) characters in a string now.

The PHP code I use for that image uploader is

[edit]
Text was too  long, I'll attach the file.

The program above will write a img.html file, that displays the uploaded image.
If you put such a ".httaccess" file in your "pix" directory, you can view all the images as I did:
Code (glbasic) Select

AddType application/x-httpd-php5       .php  
AddType application/x-httpd-php4       .php4    
AddType application/x-httpd-php5       .php5

mod_gzip_on Yes

Options +Indexes

IndexIgnore *.php *.htaccess tn*.*


See it in action:
http://www.glbasic.com/pix/


[attachment deleted by admin]


bigsofty

Cheers,

Ian.

"It is practically impossible to teach good programming style to students that have had prior exposure to BASIC.  As potential programmers, they are mentally mutilated beyond hope of regeneration."
(E. W. Dijkstra)