Vdc decompress reimagined

Looking at the C code of Pucrunch the first time and my eyes are glazing over.

I think i need to come up with rules for the encoder first before adapting it.

[Rules]
If a match is right next to current then use copy near code. since it is the smallest code byte. 256 match then if no then 2 to 64

do rle for 2 to 64 bytes. checking for 256 first

search for matches of far copy with in the 2k range. 256 then 2 to 8.
[/Rules]

Will I need a back buffer of 2048 bytes?

Work for then back to front or front to back?

I guess some people might like it, but C# just isn't for me... it is easier than C(++), but if you want easy, use VisualBASIC. VisualBASIC is present in all microsoft office apps, so learning that allows you to add very powerful customizations to your office "documents". I've made more than one "application" for a client that was nothing more than Access or Excel "document" running custom code for them. Heck, I've even had people request this type of application because it is easy for them to change (they don't want to mess with a seperate compiler or learn something more abstract like C/Java).

Your rules sound like a good start. Yes you will (eventually) need a 2K buffer. In VB you could just use either DIM M(2048) AS BYTE, if you want to manage it like an array, or DIM M$ AS STRING if you want to manage it like a string (in VB using $ for strings is optional). As a string, it is very easy to shift in new values and shift out old ones:

IF LEN(M$) >= 2048 THEN M$ = MID$(M$,2)+CHR$(new_byte) ELSE M$ = M$ + CHR$(new_byte)
Using a numeric array, however, would require a bit more work; you would either need to maintain a wrap-around pointer (MOD is your friend here), or else manually shift all the bytes.  The string method is probably what I would use (initially).

Just remember you, don't need (and shouldn't try) to do everything at once.  Start off with something stupid simple, like read in a file and write out a "compressed" version using nothing but "raw" (skip) codes.  Note this will actually INFLATE the file, but that's okay, you just want to be sure you can run it through the decompressor and get the correct result... when that works, add the RLE.  You don't need a back-buffer yet, although you will need to peek into the future for RLE.  You can use a buffer for that or possibly file-seek (umm, I can't remember ever using file-seek in VB).

When that works, add the local (sequential) copy ... this only needs 256 back-buffer.  Finally after that is working, add the long (random) copy (here you finally need 2K buffer).  And don't try to implement a 2-pass scheme yet.  Just get the fundamentals working, and verified.

Afterwords, you can optimize things.  You don't need to do full two-pass system... well not in the traditional sense of running through the whole input file twice.  You just need to memorize a choice, and determine how much that choice costs, but not actually use it yet... and then keep going as if the code was uncompressible (raw data) until you get another option to compress the data... at that time, compare if the old/saved compression technique costs less than sending raw data + the new scheme.  And choice the best one.  I can elaborate on that, but no point yet...

Once you sure it is outputting optimal compression code (for the choosen decoder rules), then along with testing of many images, you can experiment with differnt code lengths to find out which is optimal for VDC images... this is when you will learn many fascinating things about the data... what kind of compression works well and which does not (this both tend to surprise me).  This part is both easy and hard... it usually involves only trivial changes to the code for an experiment, but you need to update both the compressor and decompressor to run the experiment, and you need to run it past many images to be sure you aren't getting a false positive (or false negative).

Edit
Just thought of this: VB limits strings to either 32K or 64K in length.  Assuming it is 64K (or your image is not interlaced), you could read the entire source file into a string (M$ in the next example).  Then just 'peek' anywhere at random with

WANTED$ = MID$(M$, file_position, 1)
'or for a numeric value
BYTE_VAL = ASC(MID$(M$, file_position, 1))
Just remember it is a 1-based position, not 0-based like you would use in ML or C.

Personally, with VB, I use one of these
PRINT#n, CHR$(byte_val); 'normal file OPEN
'or using FileSystem object
OUTFILE.WRITE(CHR$(byte_val))
Just remember with PRINT# to include the the semi-colon! Of course you don't have to write out a byte at a time.  You could, for example, write a full "code sequence" stored in a string.  For example, to send 63 un-compressible bytes you might have someting like

CS$ = CHR$(63)
FOR I=0 TO 62: CS$ = CS$ + MID$(M$,file_position+I,1) : NEXT
file_position = file_position + 63
PRINT#n, CS$;
Well you can make that more effecient, by using some math to avoid a loop, but hopefully that gives a clear example.

Edit
Silly me!  The first byte should not be CHR$(63), but CHR$(63*4) to put the '00' code bits at the low end.

Module Module1

   Sub Main()
       Dim M As String
       Dim CS As String
       Dim CountValue As UShort
       Dim x As UShort
       Dim FilePosition As UShort
       Dim L As UShort
       M = My.Computer.FileSystem.ReadAllText("D:/Emulation/drive10/testdata.prg")
       FilePosition = 1
       L = Len(M)

       Do
           CountValue = 63 * 4
           CS = Chr(252)
           For I = 0 To 62
               CS = CS + Mid(M, FilePosition + I, 1)
           Next
           FilePosition = FilePosition + 63
           My.Computer.FileSystem.WriteAllText("D:/Emulation/drive10/output.prg", CS, True)
       Loop While FilePosition < L
       x = FilePosition - L
       FilePosition = L - x
       If x <> 0 Then
           CS = Chr(x * 4)
           For I = 0 To x
               CS = CS + Mid(M, FilePosition + I, 1)
           Next
           My.Computer.FileSystem.WriteAllText("D:/Emulation/drive10/output.prg", CS, True)
       End If

   End Sub

End Module
what code I have so for for VB