Author Topic: Networking for Cowards - Part 3 - HTTP POST queries  (Read 7661 times)

Offline Kitty Hello

  • code monkey
  • Administrator
  • Prof. Inline
  • *******
  • Posts: 10656
  • here on my island the sea says 'hello'
    • View Profile
Howdy folks! In the last tutorials we learned about how to interact between GLBasic programs, and GLBasic programs and a web server we wrote on our own.
This time we talk to a web server we did NOT write on our own.

Sometimes you see a webpage that offers any kind of service for you. Like weather or the current TV program. But when you click on the button, the URL does yield any of the information you queried. So, how does the webserver know what you selected in the html page?

If you look at the source code of such a website, you will see a form code like this:
Code: GLBasic [Select]
<form method="POST" action="whatever.php">

And this is the key to that door. The browser sends the request not with HTTP GET (where you see whatever.php?name=John&lastname=Doe), but in a HTTP POST request header that is hidden from the browser user. Using this technique, the browser is also able to upload larger files to the server, since the HTTP GET is limited to a few hundred characters length.

A simple HTTP request looks like this:
Code: GLBasic [Select]
    POST /somepage.php HTTP/1.0
    Content-Type: application/form-data
    Content-Length: 0


Be sure to end each line with "\r\n" and end the block with "\r\n" again.

Now, each of the <input ...> fields in the <form> tag in the html page is a variable that you must send to the server. "name" is the name of that property and "value" the value.

In order to separate the inputs and have multiline inputs, you should use a boundary string, that tells the http server when a new variable block begins.

Code: GLBasic [Select]
Content-Type: multipart/form-data, boundary=--------------XXXX1234--

Then send each variable's content like this:
Code: GLBasic [Select]
Content-Disposition: form-data; name="username"

John Doe
Pay attention! The boundary block starts with "--" and then the boundary string! So 2 more dashes at the beginning. Again, after each boundary header, add a 2nd "\r\n" to indicate the start of the value data. This way you can even send binary files:

Code: GLBasic [Select]
Content-Disposition: form-data; name="thefile"; filename="C:\foo.png"
Content-Transfer-Encoding: binary
Content-Type: image/png
Content-Type: application/octet-stream


The "image/png" just is additional information for the webserver, that could dissallow certain types for uploading. Play nice and send him was he wants.

Finally, put an emtpy boundary line at the bottom.

Make sure that the "Content-Length: " in the topmost header is set to the length of all data in bytes.

Then open a connection to that server:
Code: GLBasic [Select]
        sock% = SOCK_TCPCONNECT(server$, port%)
and send him the whole header.
Code: GLBasic [Select]
        IF SOCK_TCPSEND(sock%, send$) > 0
if it was successful, the server is starting to write you some bytes in return. What you now get is a HTTP header plus the file information the webbrowser usually displays. The header and the contents are separated by a double "\r\n" sequence.

Here's a complete example that uploads a png file to a webserver:
Code: GLBasic [Select]
// --------------------------------- //
// Project: HttpPost
// Start: Thursday, March 05, 2009
// IDE Version: 6.174

        // make a png image
        // -----------------------------------------
        DRAWRECT 0,0,240,80,RGB(0,0,200)
        PRINT "Hello World", 0,0

        GRABSPRITE 0,0,0,240,80
        SAVESPRITE "httppostimg.png", 0
        // -----------------------------------------

        LOCAL boundary$ // a boundary string that delimits the header parts
        LOCAL postdata$ // what you post

        // make a boundary string. That&#39;s a few "--" and some random characters
        boundary$ = "----------------bOunDarY-------"+INTEGER(GETTIMERALL()*1000.0)+"-"+RND(99999)+"--"

        // post variables
        INC postdata$, AddHttpPostVar$("check_password", YOUR PASSWORD HERE, boundary$)
        INC postdata$, AddHttpPostVar$("submit", "true", boundary$)
        // also upload an image -> You can adjust this function to upload whatever you want, easily.
        INC postdata$, AddHttpPostPngImage$("file[0]", "httppostimg.png", boundary$)
        // ...

        // Now get the website
        // -----------------------------------------
        // host it on a http website, using HTTP POST technique
        LOCAL server$="" YOUR SERVER HERE // the server URL
        LOCAL curl$  ="/pix/upload.php" YOUR PATH HERE // the path on the server (start with / )
        LOCAL proxy$ =""  // leave this empty for a direct connection
        LOCAL port%=3128                // use port 80 for direct connection
        LOCAL website$
        website$ = HttpPost$(proxy$, server$, curl$, port%, boundary$, postdata$)
        // -----------------------------------------

        // find the image url
        // -----------------------------------------
        LOCAL img_url$
        LOCAL pos% = INSTR(website$, "Direct Link: <a href=", 0)
        IF pos>0
                pos% = INSTR(website$, "\"", pos)
                IF pos>0
                        LOCAL endp%
                        endp% = INSTR(website$, "\"", pos+1)
                        IF endp>0
                                img_url$ = MID$(website$, pos+1, endp-pos-1)
        // -----------------------------------------

        // make a preview HTML page
        // -----------------------------------------
        OPENFILE(1, "img.html", FALSE)
        WRITELINE 1, "<html><head></head><body><img src=\""+img_url$+"\"></body></html>"
        CLOSEFILE 1
        // -----------------------------------------

// ---------------------------------------------------------------------
// Send a HTTP POST Request and return what the webbrowser would display
// ---------------------------------------------------------------------
FUNCTION HttpPost$: proxy$, server$, curl$, port%, boundary$, postdata$
LOCAL header$ // the HTTP post header

        // Fix proxy urls
        IF LEN(proxy$)
                curl$ = "http://"+server$+curl$
                server$ = proxy$

        // end post data
        postdata$ = postdata$ + "--"+boundary$+"--"

LOCAL send$
        send$ = "POST "+curl$+" HTTP/1.0\r\n"
        INC send$, "User-Agent: GLBasic\r\n"
        INC send$, "Host:"+server$+"\r\n"
        INC send$, "Accept: */*\r\n"
        INC send$, "Content-Type: multipart/form-data, boundary="+boundary$+"\r\n"
        INC send$, "Content-Length: "+LEN(postdata$) + "\r\n"
        INC send$, "\r\n"
        INC send$, postdata$

LOCAL sock%, file$
        sock% = SOCK_TCPCONNECT(server$, port%)
        IF SOCK_TCPSEND(sock%, send$) > 0
                LOCAL chunk$, rv%
                WHILE TRUE
                        rv% = SOCK_RECV(sock%, chunk$, 1024)
                        SELECT rv%
                                CASE -1
                                        DEBUG NETGETLASTERROR$()+"\n"
                                CASE -2 // non blocking socket not would block
                                        SLEEP 5
                                CASE 0
                                        BREAK // ok, reading is done
                                CASE >0
                                        INC file$, chunk$
                // jippieh! We got a response

                // split HTTP header and actual file information apart
                LOCAL pos%
                pos = INSTR(file$, "\r\n\r\n")
                IF pos>=0
                        header$ = MID$(file$, 0, pos%)
                        file$   = MID$(file$, pos%+4)

        RETURN file$

// ---------------------------------------------------------------------
// add a variable to a HTTP POST query
// ---------------------------------------------------------------------
FUNCTION AddHttpPostVar$: varname$, value$, boundary$
LOCAL postdata$
        postdata$ = "--"+boundary$+"\r\nContent-Disposition: form-data; name=\""+varname$+"\"\r\n\r\n"+value$+"\r\n"
        RETURN postdata$

// ---------------------------------------------------------------------
// upload a png image variable to a HTTP POST query
// ---------------------------------------------------------------------
FUNCTION AddHttpPostPngImage$: varname$, filepath$, boundary$
LOCAL postdata$
LOCAL content$

        IF OPENFILE(1, filepath$, TRUE)
                LOCAL b%
                WHILE ENDOFFILE(1)=FALSE
                        READBYTE 1, b%
                        // chr$(0) is a string of length "1", that contains a &#39;\0&#39; character!
                        content$ = content$ + CHR$(b%)
                CLOSEFILE 1

                postdata$ = "--"+boundary$+"\r\n"
                INC postdata$, "Content-Disposition: form-data; name=\""+varname$+"\"; filename=\""+filepath$+"\"\r\n"
                INC postdata$, "Content-Transfer-Encoding: binary\r\n"
                INC postdata$, "Content-Type: image/png\r\n"
                INC postdata$, "Content-Type: application/octet-stream\r\n\r\n" + content$ + "\r\n";

        RETURN postdata$

Beware! You need GLBasic update >= 6.184, since I fixed a bug that lets you have CHR$(0) characters in a string now.

The PHP code I use for that image uploader is

Text was too  long, I'll attach the file.

The program above will write a img.html file, that displays the uploaded image.
If you put such a ".httaccess" file in your "pix" directory, you can view all the images as I did:
Code: GLBasic [Select]
AddType application/x-httpd-php5       .php  
AddType application/x-httpd-php4       .php4    
AddType application/x-httpd-php5       .php5

mod_gzip_on Yes

Options +Indexes

IndexIgnore *.php *.htaccess tn*.*

See it in action:

[attachment deleted by admin]
« Last Edit: 2009-Mar-05 by Kitty Hello »


  • Guest
Very useful!

Offline bigsofty

  • Community Developer
  • Prof. Inline
  • ******
  • Posts: 2510
    • View Profile
Very useful!

Yes, all bookmarked for reference!  :good:


“It is practically impossible to teach good programming style to students that have had prior exposure to BASIC.  As potential programmers, they are mentally mutilated beyond hope of regeneration.”
(E. W. Dijkstra)