Author Topic: Networking - Transmitting files between two computers / peer communication  (Read 4505 times)

Offline FutureCow

  • HelpEditor
  • Prof. Inline
  • ******
  • Posts: 680
    • View Profile
I had a reason with my current project to implement file transfers between two applications. After looking at the 3 "networking for cowards" tutorials that Kitty Hello had written I found that they didn't give the exact solution to my problem. After a lot of testing (and Kitty Hello resolving a NUL character bug in the networking routines) I thought I'd try and save people having to work out how to do it themselves.

Notes for the test program
  • I've rolled both the "Server" (the file sender) and "Client" (file receiver) applications into the one program. To test the networking code you don't need two systems, you can run two copies of the program on the one computer and it will act as both client and server.
  • Port : I have arbitrarily chosen to use port 11211 on my machine, this can be anything you like (though you should stay above 1024 to avoid "reserved" ports)
  • IP address : Normally you would have some way in your program to allow someone to enter the IP address, I've hard-coded it for convenience. It's currently set to "127.0.0.1" which means the computer you're on. If you want the program to talk to another computer, you would put its IP address here
  • Filenames : Also for convenience, I've hard-coded the source and destination file names (SourceFile$ and DestFile$) at the top of the program. If you are testing both server and client on the one pc, make sure you have a different source and destination filename
  • Project type : The project has been configured as a console application, so don't forget to update your project settings so you compile a console application
  • TCP/UDP :  I've chosen to use TCP rather than UDP as TCP has guaranteed delivery of packets and is routable across the internet. As a rule UDP is quicker, but TCP is more dependable.


Program Routines
What the code does is as follows :-



Main routine[/u]
The main routine does two things, firstly it initialises the socket networking. This MUST be done before you try to send data across the network.
Code: GLBasic [Select]
SOCK_INIT()
 

Next it determines whether you want this program to act as the Server or Client, and runs the appropriate Function.



Server
The server code starts by obtaining the size of the file to be transmitted
Code: GLBasic [Select]
FileSize=GETFILESIZE(SourceFile$)
 

We try to open the file (error if it can't be opened) so we can read its contents
Code: GLBasic [Select]
FileOpened=OPENFILE(FileHandle,SourceFile$,1)
IF FileOpened=TRUE
 

Next we tell the program to listen on the socket for any incoming connection requests with the SOCK_TCPLISTEN(Socket Number) command. In my code it  assigns the name "Sock" to the socket.
Code: GLBasic [Select]
Sock=SOCK_TCPLISTEN(SocketNumber)       // Listen for connections
 

We then loop the following command continuously. The SOCK_TCPACCEPT command will return a value other than -1 when a connection attempt has been made to the open socket.
Code: GLBasic [Select]
EstablishedConnection=SOCK_TCPACCEPT(Sock,ClientIPAddress)      // Check for established client connection
IF EstablishedConnection > 0
 

We now know that we have a successfully connected client and can start talking to them. Firstly we send them the size of the file we will be transmitting. This is only needed to give them a display saying how much has been downloaded so far, and how much is still to come. We could just start writing the data at this point (assuming they knew what to do with it). In my code, it's been written so that the first thing transmitted will always be the file size. We send the data with the SOCK_TCPSEND command. It will return the number of bytes transferred to the remote end, or <0 if there was an error.
Code: GLBasic [Select]
STDOUT "Sending filesize of " + FileSize + " to client\n"
CurrentPacketSizeSent=SOCK_TCPSEND(EstablishedConnection,FileSize)              // Send size to transfer to client
 

We then wait for a confirmation message back to say the file size was received. This error checking may not end up necessary, but is good practice regardless. I've arbitrarily made my program wait to receive the confirmation string "Received-proceed". We listen for data on the same socket that we transmit on. In this instance we listen using the "SOCK_RECV" command. When it receives data the command will return either the number of bytes read, or -1
if there was an error. If it returns -2 it means the socket is not ready yet to give us data, so we sleep for 5ms before trying again. The data will be returned in the ReceiveMessage$ string.
The SOCK_RECV command will read bytes from the socket until its either read "PacketSize" number of bytes, or it encounters an "\0" character. I've configured my program to always read 1024 bytes from the socket, though again this is an arbitrary value. Without getting into how data actually travels across the network, suffice to say it's probably best to leave this under 1488 (look up MTU if you're interested) and I'd make it a fair bit smaller than that to be on the safe side.
So this is how we receive the message confirmation, and do some error checking :-
Code: GLBasic [Select]
STDOUT "Waiting on confirmation of receipt.\n"                                                  // Wait for confirmation that client received the filesize
LOCAL ReceivedMessage=0                                                                         // We haven&#39;t received the receipt yet
WHILE ReceivedMessage=0                                                                         // While we haven&#39;t had the receipt
        ConfirmationReceived=SOCK_RECV(EstablishedConnection,ReceiveMessage$,PacketSize)        // Read from the socket
        SELECT ConfirmationReceived
                CASE -1                                                                         // The socket returned an error
                        DEBUG NETGETLASTERROR$()+"\n"                                           // Print out the error message
                        BREAK                                                                  
                CASE -2                                                                         // The socket&#39;s not ready to be read
                        SLEEP 5                                                                 // Wait a little while
                CASE > 0                                                                                                                       
                        ReceivedMessage=1                                                       // We&#39;ve read some data, stop listening to socket
                        STDOUT "Message received -" + ReceiveMessage$ + ".\n"                   // Output the message received
        ENDSELECT
WEND

// When the confirmation is received...
IF ReceiveMessage$="Received-proceed"
        STDOUT "File size confirmed. Sending file data...\n"
 

We've now got the go-ahead to start sending the actual file data. I've decided that I'm going to send the data in "packets" of 1024 bytes so the first thing I do in my file reading loop is determine that there are at least 1024 bytes of data left to read. If not, I determine how many bytes are left. The result will be contained in the "Diff" variable.
Code: GLBasic [Select]
CurrentFilePos=FILEPOSITION(FileHandle)                 // Where are we in the file we&#39;re reading?
IF FileSize-CurrentFilePos < PacketSize                 // If there are less than "PacketSize" bytes left in the file
        Diff = FileSize - CurrentFilePos                // We&#39;ll make a smaller "packet" of size Diff
ELSE                                                    // Else
        Diff=PacketSize                                 // The number of chars to send (Diff) is the same as the "PacketSize"
ENDIF
 

Now we know how many bytes we're going to read in the next file read. The easiest way to grab the source data would be to do a string read of 1024 characters from the file, but unfortunately this will likely cause issues. The READSTR command used to read the data uses the NUL character (ASCII code 0 - also written as \0 or in GLBasic code as CHR$(0)) to determine when it's finished reading a string of text. Binary files unfortunately can include a NUL character as part of their standard data, so if you do a big READSTR and inadvertently read a NUL part way through, the file data will be truncated before it contains the full 1024 characters.

To get around the problem, we read the string 1 character at a time. If that character read is a NUL, I explicitly put a NUL character into the transmission string (DataString$).
Code: GLBasic [Select]
FOR Loop=1 TO Diff
        IF ENDOFFILE(FileHandle) = FALSE                        // Precaution - should never occur
                READSTR FileHandle, CurrentByte$, 1             // Read a byte from the file
                IF CurrentByte$="\0"                            // NUL read
                        INC DataString$, CHR$(0)                // Add NUL to transmission string
                ELSE                                            // Else
                        INC DataString$, CurrentByte$           // Add non-NUL character to transmission string
                ENDIF
        ELSE
                BREAK
        ENDIF
NEXT
 

We now have a data "packet" of 1024 bytes from the file (or less if there were less than 1024 bytes still to be read from the file), so we send it. We use the TCPSEND command to send the transmission string (DataString$). The value returned will be the amount of data transferred. If the value is -1 an error occurred and we need to print an error message. If no error occurred, we increment the number of bytes that have been sent (TotalAmountSent) and print out a status message.
Code: GLBasic [Select]
CurrentPacketSizeSent=SOCK_TCPSEND(EstablishedConnection,DataString$)           // Send PacketSize bytes of data
IF CurrentPacketSizeSent = -1                                                   // ERROR!
        STDOUT "ERROR! - " + NETGETLASTERROR$()+"\n"                            // Print error string
        DEBUG "ERROR! - " + NETGETLASTERROR$()+"\n"                             // Print error string
        BREAK                                                                   // Stop sending
ELSE
        INC TotalAmountSent, CurrentPacketSizeSent                              // Increment amount of data transferred
        STDOUT TotalAmountSent + " bytes of " + FileSize + " bytes sent\n"      // Output amount of data transferred
ENDIF
 

Once we've transmitted all the packets we then close the file and the socket.
Code: GLBasic [Select]
SOCK_CLOSE(Sock)                // Close socket
CLOSEFILE FileHandle            // Close file
 



Client
The client code does pretty much the reverse of the server code. When the client is receiving, the server is sending and vice-versa.
We create the connection to the server with the TCP_CONNECT command. It will return -1 if there is an error opening the connection (e.g. the server hasn't created their connection yet), or -2 if the socket isn't ready for use.
Code: GLBasic [Select]
EstablishedConnection=SOCK_TCPCONNECT(ServerIP$,SocketNumber)           // Try to connect to server
IF EstablishedConnection > -1                                                                           // Success
 

We then get the size of the file to be transmitted with the SOCK_RECV command and do error checking on the results.
Code: GLBasic [Select]
DataLength=SOCK_RECV(EstablishedConnection,ReceiveMessage$,PacketSize)  // Try to read size of file to transmit
 

Assuming the filesize was successfully read, let the server know we got it and that we're ready for the data.
Code: GLBasic [Select]
CurrentPacketSizeSent=SOCK_TCPSEND(EstablishedConnection,"Received-proceed")    // Send back a "Received-proceed" confirmation of message transmission
 

Open the destination file to be populated with the data.
Code: GLBasic [Select]
OPENFILE(FileHandle, DestFile$, 0)

Loop whilever data is being transmitted, reading data for the file with the SOCK_RECV function
Code: GLBasic [Select]
DataLength=SOCK_RECV(EstablishedConnection,ReceiveMessage$,PacketSize)  // Read "PacketSize" bytes for the destination file
 

Increment our counter of how much has been read so we can keep the user appraised of the progress, then write the data (ReceiveMessage$) to the destination file (FileHandle). When the amount of data read is less than "PacketSize", we've read all the data.
Code: GLBasic [Select]
INC ReadFileSize, DataLength                                    // We&#39;ve read valid data. Increase our count of how much has been transferred
WRITESTR FileHandle, ReceiveMessage$                            // Write transferred data to the destination file
IF DataLength < PacketSize THEN Finished=1                      // When we read less than "PacketSize" bytes, we should have all the file.
 

Display how much data was transferred, close the file and the open socket.
Code: GLBasic [Select]
STDOUT "\n\nOriginal size : "+FileSize                                                                          // Print out totals
STDOUT "\nRead size : " + ReadFileSize + "\n"
SOCK_CLOSE(EstablishedConnection)                                                                                       // Close socket
CLOSEFILE FileHandle                                                                                                                    // Close file
 

And we're done!
I've attached the complete code



So what else does the program need?
* There are two things this program needs. Firstly it could do with more error checking in a few places, reporting the error message back to the user when an error is encountered (send them the output of the NETGETLASTERROR command which works for both NET... and SOCK... commands)
* Secondly the program should also generate checksums for the file on both ends and compare them to check that the file transferred correctly.

I hope you find this useful!


[attachment deleted by admin]
« Last Edit: 2009-Nov-10 by FutureCow »

Offline Moru

  • Administrator
  • Prof. Inline
  • *******
  • Posts: 1774
    • View Profile
    • Homepage
Wow, thanks a lot, saves many hours :-) And who said programmers don't do documentation. Fine work :-)

Offline Ian Price

  • Administrator
  • Prof. Inline
  • *******
  • Posts: 4144
  • On the shoulders of giants.
    • View Profile
    • My Apps
This will come in mighty handy I'm sure. :)
I came. I saw. I played.

Offline FutureCow

  • HelpEditor
  • Prof. Inline
  • ******
  • Posts: 680
    • View Profile
Thanks for the kind remarks everyone!
There were a fair few hours work in working out how to use the networking commands and getting it all working properly (the bug with \0 characters was frustrating to find!!) Everyone here is really generous with their time when I ask questions so I was happy to be able to give something substantial back to the community for a change.
I hope you all get good use out of it!!!

Offline Kitty Hello

  • code monkey
  • Administrator
  • Prof. Inline
  • *******
  • Posts: 10697
  • here on my island the sea says 'hello'
    • View Profile
    • http://www.glbasic.com
Excellent. The next step would be to write a TFTP server/client. That's really easy to do now:
http://en.wikipedia.org/wiki/Trivial_File_Transfer_Protocol

Offline codegit

  • Dr. Type
  • ****
  • Posts: 270
    • View Profile
Dude, this is an AWESOME tutorial......thanks.  :good:
------------------------------------------
1 X Acer TravelMate 4270, laptop, XP PRO
1 X Dell Studio 17 laptop, Windows 7
1 X MacBook Pro 2,2 GHz Core 2 Duo, 2 GB RAM, 160 GB HDD, 9400M
2 X iTouch
1 X HTC Desire (Android 2.1)
iPad soon to be added

Offline Hemlos

  • To boldy go where no pixel has gone before!
  • Global Moderator
  • Prof. Inline
  • *******
  • Posts: 1634
  • Particle Hawk
    • View Profile
Awesome  :good:

This will be handy for updating my MMO game directly through the game server.

Thanks!
Volume_of_Earth(km^3) = 4/3*3.14*POW(6371.392896,3)

Offline bigsofty

  • Community Developer
  • Prof. Inline
  • ******
  • Posts: 2612
    • View Profile
This is a very nice gesture, many thanks!  :good:
Cheers,

Ian.

“It is practically impossible to teach good programming style to students that have had prior exposure to BASIC.  As potential programmers, they are mentally mutilated beyond hope of regeneration.”
(E. W. Dijkstra)