Vdc decompress reimagined

working rle now. needs some cleanup still before going on to next part.
Module Module1

   Sub Main()
       Dim outputString As String = ""
       Dim packetSize As Integer = 0
       Dim starting() As Byte = {0}
       Dim stringCount As Integer
       Dim historyPointer As Integer
       Dim processPointer As Integer
       Dim futurePointer As Integer
       Dim endOfProcess As Integer
       Dim raw As String = ""
       Dim compressTarget As String = ""
       Dim anyCompression As Boolean
       Dim count As Integer

       Dim bytes = My.Computer.FileSystem.ReadAllBytes("D:/Emulation/drive10/testdata.prg")
       'convert array into our workstring
       For index = 0 To bytes.GetUpperBound(0)
           compressTarget = compressTarget & Chr(bytes(index))
       Next
       historyPointer = 0
       processPointer = 1
       futurePointer = 256

       endOfProcess = Len(compressTarget)
       Do
           stringCount = 1
           anyCompression = False
           Do
               If Mid(compressTarget, processPointer, 1) = Mid(compressTarget, processPointer + stringCount, 1) Then
                   stringCount = stringCount + 1
                   anyCompression = True                  
                   If stringCount = futurePointer Then
                       Exit Do
                   End If
               Else
                   Exit Do
               End If
           Loop
           If anyCompression = True Then
               If Len(raw) <> 0 Then
                   outputString = outputString & Chr(Len(raw) * 4) & raw
                   raw = ""
               End If
               If stringCount = 256 Then
                   outputString = outputString & Chr(2) & Mid(compressTarget, processPointer, 1)
               Else
                   If stringCount > 64 Then
                       stringCount = 64
                   End If
                   outputString = outputString & Chr((stringCount - 1) * 4 + 2) & Mid(compressTarget, processPointer, 1)
               End If
               processPointer = processPointer + stringCount
               If processPointer > endOfProcess Then
                   outputString = outputString & Chr(0)
                   Exit Do
               End If
           End If
           If anyCompression = False Then
               raw = raw & Mid(compressTarget, processPointer, 1)
               processPointer = processPointer + 1
               If Len(raw) = 63 Then
                   outputString = outputString & Chr(63 * 4) & raw
                   raw = ""
               End If
               If processPointer > endOfProcess Then
                   If Len(raw) = 0 Then
                       outputString = outputString & Chr(0)
                       Exit Do
                   End If
                   outputString = outputString & Chr(Len(raw) * 4) & raw & Chr(0)
                   Exit Do
               End If
           End If
           If processPointer + futurePointer > endOfProcess Then
               futurePointer = futurePointer - (processPointer + futurePointer - endOfProcess)
           End If
           If futurePointer = 0 Then
               outputString = outputString & Chr(0)
               Exit Do
           End If
       Loop
       Dim outputFile(Len(outputString)) As Byte
       For count = 1 To Len(outputString)
           outputFile(count) = Asc(Mid(outputString, count, 1))
       Next
       My.Computer.FileSystem.WriteAllBytes("D:/Emulation/drive10/outputData.prg", starting, False)
       My.Computer.FileSystem.WriteAllBytes("D:/Emulation/drive10/outputData.prg", outputFile, True)
   End Sub

End Module


The codes are valid to my decoder.



So many bodies, so little time to hide them.

Great to hear that hear RLE is verified working. Personally, I would try to clean up / simplify the code before moving on to Copy. Well you could just move on to Copy, and once you get it working it should be rather obvious how you can "optimize" the code to deal with either RLE/Copy in a generic/effecient manner. Ummm, yeah, there are multiple ways to develope a "sophisticated" piece of software... the end result is all that really matters:
1. Does it do the job ?
2. Does it do it always do it correctly?
3. Is it effecient?
4. Is it easy to use?

Feel free to re-order those... in particular #4 should be much higher in the list from a practical/human/user point of view... in fact, I think many software publishers use that formula!!! In other words, they often put human usability last... which is just plain wrong in my opinion, and why I have little respect for modern software. (In general... sometimes you find a gem that is extremely user friendly.)

Glad that you got it working! You might want to add some REMs for everyone's benefit, especially yourself a year from now

I think I understand it... so first it does three checks, first a 'big' copy of 256, then a 'normal' copy of 64 or less, then (if both those fail), a check for RLE. Next it just acts on the discovery... it outputs a 'big' or 'normal' copy packet (anyCompression=2), or an RLE packet (anyCompression=1), or else it just builds up a raw packet (anyCompression=0).

Well that is my generic understanding... of course along the way you have to (virtually) write out packets (advance pointers), and watch for the end-of-file... and other such annoying details which make programming a chore. And of course at the end you write out the CBM load address and (really write) all the packets (outputFile) to the target file (outputData.prg).

Anyway, if you are confident it is encoding and decoding correctly, you can start testing some real images to see what type of compression you get. I would be very interested to know what you get. My video encoder uses very similar technique, and on VIC-II images it typically gets 50% or more... but that is due to the nature of video (a lot of data is redundant anyway) and also the sloppy / approximate image generation that I use (the approximation or "lossy" encoding improves compression). Because you aren't doing lossy compression I imagine compression would only be around 25% ~ 33% for reasonably complex images... in other words, a compression of 50% or more wouldn't normally happen unless the image was extremely "boring".

This is just a guess (based on "new scenes" in my VIC-II videos), but I would really love to hear what you get! I imagine there are major differences in how data compresses between the VIC and VDC images. So, keep us posted.