|
|
Post by VDC 8x2 on Sept 1, 2014 2:23:41 GMT
The two things I am shooting for is speed on the c-128 and saving bytes that have to be transferred to the vdc.
|
|
|
|
Post by hydrophilic on Sept 28, 2014 8:12:31 GMT
I have Win8, now, so I can test 64-bit code no problem... still not sure what the issue was with 32-bit. I *really* need to finish my VDC video encoder... I've got Vector Quantization working, but I'm having issues with constraining the colors for VDC text mode (i.e., 1 global color + 1 unique color and/or Reverse font) for each cell.
So far, the best results occur when I limit the 'global' color to an achromatic value (black, some shade of gray, or white). It works fine (and with much better quality) when using real bitmap mode (any 2 colors per cell), but that is just way too much data to push through $d601 for a video (thus, I use text mode with custom font).
|
|
|
|
Post by VDC 8x2 on Sept 28, 2014 18:01:54 GMT
Just a wild idea...
have the values 0 to 255 stored in vdc memory then just copy them around where needed?
maybe more work then just pushing the data... hmm.
|
|
|
|
Post by hydrophilic on Sept 29, 2014 3:37:49 GMT
The video encoder does some tricks to maximize what can be copied (or skipped). Because you can assign any screen code to a custom font character, it helps to re-order the codes of the new/next image to match the codes of the current/previous image. If a block of video is in the same spot, it can be skipped (never write the ram) if it moved left or up, a simple copy can be done. If it moved right or down (on the same screen), its a bit more complicated... it needs to get copied before it is over-written with new data that is left/above.
Err, guess I didn't explain it well (need some diagrams), but I think it is they same idea you're talking about... copying a known / pre-set block of bytes instead of uploading new values.
Yeah, I need to do some threshold tests to see which sizes are worth moving as opposed to simply uploading new data.
For (VIC) bitmap mode, I had a similar idea to write some common/useful byte sequences to unused areas or RAM and then copy them whenever needed... they would always be available and not get erased as new images are loaded. It was an idea that I never implemented or tested! ...because VIC access is quick and the common/useful byte sequences are often in memory anyway (except at the start of the video).
For your purposes (single image), I don't think this would be practical... you would have to upload the byte sequence into the VDC at least once, and if it has already been loaded, the normal Copy methods you already use should work their magic.
Oh yeah, I assume you are talking about a byte sequence like $55,$aa,$55,$aa or $FF,$aa,$55,$00 ... trying to copy 1 byte from an array of 0,1,2,3,4...255 would almost surely be slower than uploading a single byte.
|
|
|
|
Post by hydrophilic on Oct 3, 2014 8:17:58 GMT
Bump! Have you made in progress or maybe decided the current version is "good enough"? Not that I am rushing you... my own code is still not Release (or even Beta) quality  |
|
|
|
Post by VDC 8x2 on Oct 3, 2014 19:58:50 GMT
I haven't got a chance to get to it lately. I am going to finish it up and release the parallel beta soon.
|
|
|
|
Post by hydrophilic on Jan 23, 2015 8:20:10 GMT
Bump! Any progress?
|
|
|
|
Post by VDC 8x2 on Jan 25, 2015 1:25:50 GMT
Reviewing my notes then I should be able to finish the multi-threading version this week.
|
|
|
|
Post by hydrophilic on Jan 27, 2015 7:14:16 GMT
Cool... I have only *indirectly* played with multi-threading... in other words, I have tested multi-thread code works on a single-core CPU, but until now I did not have a multi-core CPU to test. ANYWAY, I look forward to testing your software and providing feedback. (I'll be nice  ) |
|
|
|
Post by VDC 8x2 on Jan 28, 2015 1:30:51 GMT
Any suggestions for improving the ui? Here is the code. The inner three loops now run in parallel. Imports System.Threading.Tasks
Public Class VDCompress
Dim outputString As String = ""
Dim starting() As Byte = {0}
Dim historyPointer As Integer = 0
Dim processPointer As Integer = 1
Dim futurePointer As UInteger = 256
Dim endOfProcess As Integer
Dim updatePointer As UInteger
Dim farSize As Integer
Dim raw As String = ""
Dim compressTarget As String = ""
Dim anyCompression(3) As UInteger
Dim stringCount(3) As UInteger
Dim index As UInteger
Private Sub loadImage_Click(sender As Object, e As EventArgs) Handles loadImage.Click
OpenFileVDC.InitialDirectory = "C:\"
Do
OpenFileVDC.ShowDialog()
If OpenFileVDC.FileName <> "" Then
Exit Do
End If
Loop
Dim bytes = My.Computer.FileSystem.ReadAllBytes(OpenFileVDC.FileName)
'convert array into our workstring
For Me.index = 0 To bytes.GetUpperBound(0)
compressTarget = compressTarget & Chr(bytes(index))
Next
endOfProcess = Len(compressTarget)
End Sub
Private Sub compressImage_Click(sender As Object, e As EventArgs) Handles compressImage.Click
'start of the encoding loop
Me.CompressionProgress.Minimum = 0
Me.CompressionProgress.Maximum = Me.endOfProcess
Me.CompressionProgress.Value = 0
Do
'making 1 , 2 and 3 to compete against each other
Parallel.For(1, 3, Sub(index)
stringCount(index) = 1
anyCompression(index) = 0
If index = 2 Then
'256 mirror copy
If anyCompression(2) = 0 Then
If historyPointer > 257 Then
If futurePointer = 256 Then
stringCount(2) = futurePointer
If Mid(compressTarget, processPointer - 257, stringCount(2)) = Mid(compressTarget, processPointer + 256, stringCount(2)) Then
anyCompression(2) = stringCount(2)
End If
End If
End If
End If
'64 mirror copy
If anyCompression(2) = 0 Then
If historyPointer > 65 Then
If futurePointer > 63 Then
stringCount(2) = 64
Else
stringCount(2) = futurePointer
End If
Do
If Mid(compressTarget, processPointer - stringCount(2) - 1, stringCount(2)) = Mid(compressTarget, processPointer + stringCount(2), stringCount(2)) Then
anyCompression(2) = stringCount(2)
Exit Do
Else
stringCount(2) = stringCount(2) - 1
If stringCount(2) = 1 Then
Exit Do
End If
End If
Loop
End If
End If
End If
If index = 3 Then
'256 far copy
If anyCompression(3) = 0 Then
If futurePointer > 255 And historyPointer > 255 Then
farSize = 256
stringCount(3) = historyPointer
Do
If Mid(compressTarget, processPointer - stringCount(3), farSize) = Mid(compressTarget, processPointer, farSize) Then
anyCompression(3) = farSize
Exit Do
Else
stringCount(3) = stringCount(3) - 1
If stringCount(3) < 257 Then 'nothing found so set stringCount for the next check
Exit Do
End If
End If
Loop
End If
End If
' 64 and under far copy
If anyCompression(3) = 0 Then
If futurePointer > 2 And historyPointer > 2 Then
If futurePointer < 64 Then
farSize = futurePointer
Else
farSize = 64
End If
Do
stringCount(3) = historyPointer
Do
If Mid(compressTarget, processPointer - stringCount(3), farSize) = Mid(compressTarget, processPointer, farSize) Then
anyCompression(3) = farSize
Exit Do
Else
stringCount(3) = stringCount(3) - 1
If stringCount(3) < farSize + 1 Then 'nothing found so set stringCount for the next check
Exit Do
End If
End If
Loop
If anyCompression(3) > 0 Then
Exit Do
End If
farSize = farSize - 1
If farSize = 2 Then 'if less than 3 then get out of hear no compression
Exit Do
End If
Loop
End If
End If
End If
If index = 1 Then
'rle for 256 or 64 bytes
If anyCompression(1) = 0 Then
Do
If Mid(compressTarget, processPointer, 1) = Mid(compressTarget, processPointer + stringCount(1), 1) Then
stringCount(1) = stringCount(1) + 1
anyCompression(1) = 1
If stringCount(1) = futurePointer Then
Exit Do
End If
Else
Exit Do
End If
Loop
If anyCompression(1) > 0 Then
If stringCount(1) > 64 And stringCount(1) < 256 Then 'out of bounds so truncate it
anyCompression(1) = 64
stringCount(1) = 64
Else
anyCompression(1) = stringCount(1)
End If
End If
End If
End If
End Sub)
'if any compression happened then flush raw buffer
If anyCompression(1) > 0 Or anyCompression(2) > 0 Or anyCompression(3) > 0 Then
If Len(raw) <> 0 Then
outputString = outputString & Chr(Len(raw) * 4) & raw
raw = ""
End If
'process 256 far copy
If anyCompression(3) >= anyCompression(1) And anyCompression(3) >= anyCompression(2) Then
'Console.WriteLine("far copy " & anyCompression(3))
If anyCompression(3) = 256 Then 'nnnnnnxx with xx =11 lllllsss with high byte low byte
outputString = outputString & Chr(3) & Chr((stringCount(3) And 65280) / 256) & Chr(stringCount(3) And 255)
updatePointer = anyCompression(3)
Else
outputString = outputString & Chr(((anyCompression(3) - 1) * 4) + 3) & Chr((stringCount(3) And 65280) / 256) & Chr(stringCount(3) And 255)
updatePointer = anyCompression(3)
End If
ElseIf anyCompression(2) >= anyCompression(1) Then
'process mirror packet
'Console.WriteLine("mirror " & anyCompression(2))
If anyCompression(2) = 256 Then
outputString = outputString & Chr(1)
updatePointer = 256
Else
outputString = outputString & Chr((anyCompression(2) - 1) * 4 + 1)
updatePointer = anyCompression(2)
End If
Else
'process rle packet
'Console.WriteLine("rle " & anyCompression(1))
If anyCompression(1) = 256 Then
outputString = outputString & Chr(2) & Mid(compressTarget, processPointer, 1)
updatePointer = 256
Else
outputString = outputString & Chr(anyCompression(1) - 1 * 4 + 2) & Mid(compressTarget, processPointer, 1)
updatePointer = anyCompression(1)
End If
End If
Else
raw = raw & Mid(compressTarget, processPointer, 1)
updatePointer = 1
'Console.WriteLine("raw")
'if raw has reached 63 bytes then flush the buffer to output and clear
If Len(raw) = 63 Then
outputString = outputString & Chr(63 * 4) & raw
raw = ""
End If
End If
historyPointer = historyPointer + updatePointer
If historyPointer > 65535 Then
historyPointer = 65535
End If
processPointer = processPointer + updatePointer
Me.CompressionProgress.Value = processPointer
If processPointer + futurePointer > endOfProcess Then
futurePointer = futurePointer - (processPointer + futurePointer - endOfProcess)
End If
If futurePointer = 0 Then
'if any raw left in buffer then flush out before ending
If Len(raw) <> 0 Then
outputString = outputString & Chr(Len(raw) * 4) & raw
raw = ""
End If
outputString = outputString & Chr(0)
Exit Do
End If
Loop
Me.CompressionProgress.Value = Me.endOfProcess
End Sub
Private Sub saveCompressedFile_Click(sender As Object, e As EventArgs) Handles saveCompressedFile.Click
Me.CompressionProgress.Value = 0
Dim outputFile(Len(outputString)) As Byte
For Me.index = 1 To Len(outputString)
outputFile(index) = Asc(Mid(outputString, index, 1))
Next
SaveFileVDC.InitialDirectory = "C:\"
SaveFileVDC.FileName = OpenFileVDC.FileName & ".vdcp"
SaveFileVDC.ShowDialog()
My.Computer.FileSystem.WriteAllBytes(SaveFileVDC.FileName, starting, False)
My.Computer.FileSystem.WriteAllBytes(SaveFileVDC.FileName, outputFile, True)
End Sub
End Class
Attachments:VDCmpress-parallel.zip (9.06 KB)
|
|
)