Transmitting files across the network

[FutureCow]

Anyone have any ideas? I've tried this both reading strings and reading UBytes, and the client seems to stop reading data when it hits a \0 value. When I get this sorted I'll post it as a code example on the forum.

The source code wis set up to run on one pc (ie. IP address 127.0.0.1) - you'll need to run 2 copies, one for the client, one for the server.
Change the SourceFile to reflect a valid file on your Server system, and DestFile to where you'd like the copy put on the Client.

Code (glbasic) Select Expand


GLOBAL SourceFile$
GLOBAL DestFile$
SourceFile$="c:/glbasic/outfile.txt"
DestFile$="c:/glbasic/infile.txt"

// ------------------------------------------------------------- //
// ---  MAIN  ---
// ------------------------------------------------------------- //

SOCK_INIT()
LOCAL quit=0
WHILE quit=0
STDOUT "(C)lient / (S)erver / (Q)uit\n"
Result$ = INKEY$()
IF Result$="C" OR Result$="c"
GOSUB ClientSub
ENDIF
IF Result$="s" OR Result$="S"
GOSUB ServerSub
ENDIF
IF Result$="q" OR Result$="Q"
quit=1
ENDIF
WEND


// ------------------------------------------------------------- //
// ---  CLIENT  ---
// ------------------------------------------------------------- //
SUB ClientSub:
STDOUT "Running as client!\n"
LOCAL FileSize%
LOCAL ReceiveMessage$
LOCAL ReadFileSize=0 // How much data has been read so far
LOCAL FileHandle
FileHandle=GENFILE()

EstablishedConnection=SOCK_TCPCONNECT("127.0.0.1",11211) // Try to connect to server
IF EstablishedConnection > -1 // Success

Finished=0
WHILE Finished=0
DataLength=SOCK_RECV(EstablishedConnection,ReceiveMessage$,1024) // Try to read size of file to transmit
SELECT DataLength
CASE -1
DEBUG NETGETLASTERROR$()+"\n"
BREAK
CASE -2 // non blocking socket not would block
SLEEP 5
DEFAULT
Finished=1
ENDSELECT
WEND

Finished=0
FileSize%=ReceiveMessage$

OPENFILE(FileHandle, DestFile$, 0) // Open file for writing

SLEEP 200 // Let some data buffer
WHILE Finished=0
DataLength=SOCK_RECV(EstablishedConnection,ReceiveMessage$,1024) // Read 1024 bytes for the destination file
SELECT DataLength
CASE -1
DEBUG NETGETLASTERROR$()+"\n"
BREAK
CASE -2 // non blocking socket not would block
SLEEP 5
CASE >0
INC ReadFileSize, DataLength // Increase our count of how much has been transferred
WRITESTR FileHandle, ReceiveMessage$ // Write transferred data to the destination file

IF DataLength < 1024 THEN Finished=1 // When we read less than 1024 bytes, we've got all of them
STDOUT "\nRead " + ReadFileSize + " bytes of " + FileSize + " total."
ENDSELECT
WEND
STDOUT "\n\nOriginal size : "+FileSize
STDOUT "\nRead size : " + ReadFileSize
SOCK_CLOSE(EstablishedConnection)
ELSE
ErrorMsg$=NETGETLASTERROR$()
STDOUT "Error message = " + ErrorMsg$ + "\n"
ENDIF
CLOSEFILE FileHandle
ENDSUB // CLIENT






// ------------------------------------------------------------- //
// ---  SERVER  ---
// ------------------------------------------------------------- //
SUB ServerSub:
STDOUT "Server!\n"

LOCAL Done = 0
LOCAL ClientIPAddress%
LOCAL TransmitMessage$
LOCAL FileHandle
LOCAL FileName$
LOCAL DataByte%
LOCAL DataString$
LOCAL FileSize
LOCAL AmountSent=0

FileSize=GETFILESIZE(SourceFile$)

FileHandle=GENFILE()
FileOpened=OPENFILE(FileHandle,SourceFile$,1)
IF FileOpened=TRUE

DataString$=""

Sock=SOCK_TCPLISTEN(11211) // Listen for connections
STDOUT "Waiting for connection...\n"
WHILE Done=0
EstablishedConnection=SOCK_TCPACCEPT(Sock,ClientIPAddress) // Try to connect to a client
IF EstablishedConnection <> -1 // Create connection
STDOUT "Established connection...\n"
STDOUT "Sending filesize of " + FileSize + " to client\n"

SOCK_TCPSEND(EstablishedConnection,FileSize) // Send size to transfer to client

STDOUT "Sending file data...\n"
// Transmit file

WHILE ENDOFFILE(FileHandle) = FALSE
Done=1
READSTR FileHandle, DataString$, 1024 // Read 1024 bytes of the file
HasBeenSentFlag=0 // Reset flag to ensure files ending on 1024 byte boundaries don't get extra data sent
SOCK_TCPSEND(EstablishedConnection,DataString$) // Send 1024 bytes of data

INC AmountSent, 1024
STDOUT AmountSent + " bytes of " + FileSize + " bytes sent\n"
HasBeenSentFlag=1 // To ensure files ending on 1024 byte boundaries don't get an extra data packet sent
WEND
IF HasBeenSentFlag=0
SOCK_TCPSEND(EstablishedConnection,DataString$) // send last block of <1024 bytes of data
STDOUT "="
ENDIF
ENDIF
WEND
ENDIF
SOCK_CLOSE(Sock)
CLOSEFILE FileHandle
ENDSUB // SERVER





Testing it on a 300K txt file, it also seems to miss both the first and last 1Kb, but I'll figure that bug out later.

[FutureCow]

Ocean,
You have to have them both using the same port number - if they were on different ports they wouldn't be connecting to each other. It works as is for one way communication, trying to get two way working at the moment.
According to the TCP_ACCEPT man page :-

You can now communicate with other processes by reading from or writing to this socket.

That implies two way communication on the one socket.

[FutureCow]

I have 2 way communication on the one socket, but issues with chr$(0) characters.
The following code transfers a source file on the server machine to a destination file on the client machine. Currently filenames and IP are hardcoded at the top of the file for testing purposes.

When run using localhost as both source and destination, binary files transfer correctly. When run on a LAN using 2 separate PCs, chr$(0) characters caused my ~300K test .exe to end up a ~7K file. ASCII files transfer correctly. Swapping the following lines around

Code (glbasic) Select Expand

                           INC DataString$, CHR$(0)
//                           INC DataString$, "a"
to do a string replacement of chr$(0) with "a" in the program result in the full size file being created (but obviously it's now corrupt).
Any ideas how I can get chr$(0) to transfer? I've tried reading/writing whole 1024byte strings, reading the file one character at a time (as per the code below) and reading the file as ubytes - none of the solutions work.
I've got no idea why it works when both server and client are the same machine either.

Code (glbasic) Select Expand

GLOBAL SourceFile$, DestFile$, ServerIP$, SocketNumber

SourceFile$="c:/glbasic/source.exe"
DestFile$="c:/temp/destination.exe"
ServerIP$="127.0.0.1"
SocketNumber=11211

// ------------------------------------------------------------- //
// ---  MAIN  ---
// ------------------------------------------------------------- //

SOCK_INIT()
LOCAL quit=0
WHILE quit=0
STDOUT "(C)lient / (S)erver / (Q)uit\n"
Result$ = INKEY$()
IF Result$="C" OR Result$="c"
GOSUB ClientSub
ENDIF
IF Result$="s" OR Result$="S"
GOSUB ServerSub
ENDIF
IF Result$="q" OR Result$="Q"
quit=1
ENDIF
WEND


// ------------------------------------------------------------- //
// ---  CLIENT  ---
// ------------------------------------------------------------- //
SUB ClientSub:
STDOUT "Running as client!\n"
LOCAL FileSize%
LOCAL ReceiveMessage$
LOCAL ReadFileSize=0 // How much data has been read so far
LOCAL FileHandle
FileHandle=GENFILE()

// Connect to server
EstablishedConnection=SOCK_TCPCONNECT(ServerIP$,SocketNumber) // Try to connect to server
IF EstablishedConnection > -1 // Success
Finished=0
WHILE Finished=0
DataLength=SOCK_RECV(EstablishedConnection,ReceiveMessage$,1024) // Try to read size of file to transmit
SELECT DataLength
CASE -1
DEBUG NETGETLASTERROR$()+"\n"
BREAK
CASE -2 // non blocking socket not would block
SLEEP 5
DEFAULT
Finished=1
ENDSELECT
WEND

// Send back confirmation of message receipt
FileSize%=ReceiveMessage$
STDOUT "\n\nReceived file size to transfer : "+FileSize
STDOUT "\nSending receipt of confirmation.\n"
CurrentPacketSizeSent=SOCK_TCPSEND(EstablishedConnection,"Received-proceed")
STDOUT "CurrentPacketSizeSent = " + CurrentPacketSizeSent + "\n"

// Receive data loop
STDOUT "Message Sent. Waiting for file data.\n" // Now receive data
OPENFILE(FileHandle, DestFile$, 0) // Open destination file for writing
Finished=0

WHILE Finished=0
DataLength=SOCK_RECV(EstablishedConnection,ReceiveMessage$,1024) // Read 1024 bytes for the destination file
SELECT DataLength
CASE -1
DEBUG NETGETLASTERROR$()+"\n"
BREAK
CASE -2 // non blocking socket not would block
SLEEP 5
CASE >0
INC ReadFileSize, DataLength // Increase our count of how much has been transferred
WRITESTR FileHandle, ReceiveMessage$ // Write transferred data to the destination file
IF DataLength < 1024 THEN Finished=1 // When we read less than 1024 bytes, we've got all of them
IF DataLength = 1 AND ReceiveMessage$="\0"
DEC ReadFileSize,1 // Ignore EOF marker
ENDIF
STDOUT "\nRead " + ReadFileSize + " bytes of " + FileSize + " total."

CASE 0
Finished=1
ENDSELECT
WEND

STDOUT "\n\nOriginal size : "+FileSize
STDOUT "\nRead size : " + ReadFileSize + "\n"
SOCK_CLOSE(EstablishedConnection) // Close socket
ELSE
ErrorMsg$=NETGETLASTERROR$()
STDOUT "Error message = " + ErrorMsg$ + "\n"
ENDIF
CLOSEFILE FileHandle // Close file
ENDSUB // CLIENT






// ------------------------------------------------------------- //
// ---  SERVER  ---
// ------------------------------------------------------------- //
SUB ServerSub:
STDOUT "Server!\n"

LOCAL Done = 0
LOCAL ClientIPAddress%
LOCAL TransmitMessage$
LOCAL FileHandle
LOCAL FileName$
LOCAL DataByte%
LOCAL DataString$
LOCAL FileSize
LOCAL TotalAmountSent=0,CurrentPacketSizeSent=0
LOCAL CurrentPacketCounter
LOCAL Loop
LOCAL CurrentByte$
LOCAL Diff
LOCAL ReceiveMessage$

FileSize=GETFILESIZE(SourceFile$)

FileHandle=GENFILE()
FileOpened=OPENFILE(FileHandle,SourceFile$,1)
DataString$=""

IF FileOpened=TRUE

// Wait for connection
Sock=SOCK_TCPLISTEN(SocketNumber) // Listen for connections
STDOUT "Waiting for connection...\n"
WHILE Done=0
EstablishedConnection=SOCK_TCPACCEPT(Sock,ClientIPAddress) // Establish client connection

IF EstablishedConnection > 0 // If connection was successful
Done=1
STDOUT "Established connection...\n"
STDOUT "Sending filesize of " + FileSize + " to client\n"

CurrentPacketSizeSent=SOCK_TCPSEND(EstablishedConnection,FileSize) // Send size to transfer to client
IF CurrentPacketSizeSent = -1 // ERROR!
STDOUT "ERROR! - " + NETGETLASTERROR$()+"\n"
DEBUG "ERROR! - " + NETGETLASTERROR$()+"\n"
BREAK
ENDIF

STDOUT "Waiting on confirmation of receipt.\n" // Wait for confirmation that client received the filesize
LOCAL ReceivedMessage=0
WHILE ReceivedMessage=0
ConfirmationReceived=SOCK_RECV(EstablishedConnection,ReceiveMessage$,1024)
SELECT ConfirmationReceived
CASE -1
DEBUG NETGETLASTERROR$()+"\n"
BREAK
CASE -2 // non blocking socket not would block
SLEEP 5
CASE > 0
ReceivedMessage=1
STDOUT "Message received -" + ReceiveMessage$ + ".\n"
ENDSELECT
WEND


// When the confirmation is received...
IF ReceiveMessage$="Received-proceed"
STDOUT "File size confirmed. Sending file data...\n"

// Transmit file loop
WHILE ENDOFFILE(FileHandle) = FALSE
DataString$=""
CurrentFilePos=FILEPOSITION(FileHandle)
IF FileSize-CurrentFilePos < 1024 // If there's less than 1024 bytes left in the file
Diff = FileSize - CurrentFilePos // We'll make a smaller "packet"
ELSE
Diff=1024
ENDIF

// Prepare 1024 byte data "packet"
FOR Loop=1 TO Diff
IF ENDOFFILE(FileHandle) = FALSE
READSTR FileHandle, CurrentByte$, 1 // Read a byte from the file
IF CurrentByte$="\0"
INC DataString$, CHR$(0)
// INC DataString$, "a"
ELSE
INC DataString$, CurrentByte$ // Assemble it into our 1024 byte transfer packet
ENDIF
ELSE
BREAK
ENDIF
NEXT

// Send data "packet" of 1024 bytes
CurrentPacketSizeSent=SOCK_TCPSEND(EstablishedConnection,DataString$) // Send 1024 bytes of data
IF CurrentPacketSizeSent = -1 // ERROR!
STDOUT "ERROR! - " + NETGETLASTERROR$()+"\n"
DEBUG "ERROR! - " + NETGETLASTERROR$()+"\n"
BREAK
ELSE
INC TotalAmountSent, CurrentPacketSizeSent // Keep track of amount of data transferred
STDOUT TotalAmountSent + " bytes of " + FileSize + " bytes sent\n"
ENDIF
ENDIF
// Send EOF to close remote connection
DataString$=CHR$(0)
CurrentPacketSizeSent=SOCK_TCPSEND(EstablishedConnection,DataString$) // Send EOF to close connection
WEND
ENDIF
WEND
ENDIF
SOCK_CLOSE(Sock) // Close socket
CLOSEFILE FileHandle // Close file
ENDSUB // SERVER




[MrTAToad]

It looks like a bug if CHR$(0) isn't being transmitted.  What you replace it with would depend on whether other value will be used -255 is usually a good value to use as an alternative.

When testing, testing network stuff on one machine is fraught with additional problems, not least the programs speed.  I have also found that the programs tend to fight against each other (with the active program generally get both host and client messages) which cause additonal problems.  If you can, try multiple machines.

[FutureCow]

It looks like a bug if CHR$(0) isn't being transmitted.  What you replace it with would depend on whether other value will be used -255 is usually a good value to use as an alternative.

I'll post it as a bug in the bug forum and see what Gernot says.
I don't see how you could actually replace it with anything. In a binary file, each byte can only be one of 256 characters, and as far as I'm aware, every character is used (well, at least is likely to be used) in for example an exe file - therefore you can't replace it with anything. The only way I can see that you could replace it would be to encode every byte as a 2 byte sequence instead, but that means double the amounts of bytes transmitted for binary data. I don't want to get into more complex encoding schemes (or any encoding schemes actually!!!)

Other than the CHR$(0) issue, I found the program worked with itself without any issues - I agree that trying on multiple machines is wise though (which is when I found the "bug" )

[Kitty Hello]

Yes, it's a bug. True shame.

[FutureCow]

Is there any workaround? 

[MrTAToad]

Possibly UUEncode the data into ASCII characters ?

Or encode numbers 0 to 255 as a 16-bit number ?

[FutureCow]

Possibly UUEncode the data into ASCII characters ?

Or encode numbers 0 to 255 as a 16-bit number ?

Good idea - that's pretty much what I put in my post in the bug forum. I was hoping for a GLBasic command solution though   

[Kitty Hello]

Bugfix in next update.