MTR 6 ANNOUNCEMENTS - READ FIRST!

[rthpjm]

Massive breakthrough... im smiling.

I say no more as there is still a way to go ...

GB, now you’re just teasing us.

[geezerbuttz]

7 complex routines to write , on the third now.

[geezerbuttz]

ok , got the 7 down to 2 with some crafty optimisations...

Still have to do the final structure update that MTR needs from this new information, and run the tests of course...
( then i have to tear out the redundant code left over)

I will , most likely , once this is validated , crowbar it into DVDSector probe , as it gives better results on finding the bups in 'layered' sets.

more soon...

[GrizzlyHippo]

GB, quick question(s), and slightly off topic so forgive me and any naivety.

Will MTR 6 work with BluRay disks? I ask as over time our TV has gotten larger and larger and now ripped DVDs just don't have brilliant clarity when viewed on a 65" TV.

If it will, can you recommend an external BluRay drive for use with an M1 MacbookPro?

Oh, and you are a beast with all this work!

Cheers,
Tom

[geezerbuttz]

nope , never will.

The returns for the amount of work you need to do to keep it current is simply not worth it.
Maybe when I move on from MTR , i can devote my time to it then , but until then ,

see above.

[GrizzlyHippo]

Oh well, thanks for the fast reply.

[TheSystemHasFailed]

nope , never will.

The returns for the amount of work you need to do to keep it current is simply not worth it.
Maybe when I move on from MTR , i can devote my time to it then , but until then ,

see above.

Honestly I would prefer that MTR be the best it can be with respect to DVD protection schemes and never see it work with BluRay. There are other tools out there for BluRay and frankly there are a plethora of DVD still being manufactured that need a tool like MTR.

[geezerbuttz]

All new is in , all old is out.

I had to make some changes how ifo's are handled , as the old code was reliant on data that is not yet available ,
Thats done.

Im now tuning the code , looking for areas of optimisation etc.

fixing bugs and conditions as they arise.

Its a lot of changes , so , it will take a few days before it becomes completely stable.

found a condition where the ISO has encrypted data on it.. ( old extraction )
In MF mode , this kills the subtitle rebuild , as the read values for the bits in the stream are all wrong, so a check was added.
Also , if the ColorTables are NULL -> we now return a generic color , just so we dont crash and burn.

( oh yes , its a lot faster loading disks )
More soon.


its all go otherwise...

[geezerbuttz]

while walking today , getting some much needed fresh air, i deduced another way that the disk sectors can be obfuscated.

While I have not seen this method , it is most certainly possible. ( im not going to draw a picture for them here )

Any way , a few extra updates to the building code , and we have dealt with that.
I now need to change the final stages of the rebuilder to process the extra data that I now produce to counter
this possible structural protection.

That will be done tomorrow.

I still have to update some of the IFO reading routines as they were dependent on code that no longer exists.
Good really as it was a hack to get those working... better to pass blocks to classes that need the extra functionality.

More soon...

[geezerbuttz]

been otimising the new code , removing more redundant stuff.
Build a corrected disk mirror that we now access form the back end of the app.

Front end is 90% replaced.

Debugging.

So far , DVDRebuild has issues .
Will get to that next.
G

[geezerbuttz]

ok , i corrected the issues.
Last one was a bit of a bugger to find , but in the end it was down to stupidity.

I just hammered the app , with multiple extractions , multiple types , using the complex disks with issues that I was sent.
The new front end worked well , no crashes , no bad extractions...

I do need a few days testing , a lot changed over the last week...

G

[geezerbuttz]

A side note:

I reworked the the MF user interface to stay in sync with the main volume menu.
Far better, and makes it easier to maintain a connection between the two selection methods.

[geezerbuttz]

looking at improving the speed of bup hunting..

Done MTR , but there will be cases where it has to nibble away to find them , I have a few more ideas.

Re DVDSector probe , not acceptable ...
I will upgrade the code to help speed up the finding the location of the little buggers....

G

[geezerbuttz]

oh , I forgot:

I added the bulk loading methods to MTR , so if you have multiple disks loaded and you want all to be ISo etc...,
you can select the ISO bulk loader to load all at the same time..

Just a convenience method realy.

[geezerbuttz]

Hi All,

I have been developing a robust BUP reader to get into Probe and MTR.

On a normal disc , the BUP is located in the last corresponding sectors at the end of the title set.
That way , if the IFO fails , the player can easily located its backup.

On discs these days , we have to contend with ifos that have deliberate errors in them that invalidate the end of the titleset address,
Bups that are simply missing , or located in a non standard place.

While running tests on ISO is not as issue as the speed i good , locating by sector scanning on a genuine disk can be painfully slow.

I need some feedback on how to best handle this,.

Should we limit the check to looking to where the bup should be , perhaps extending the check to the end of the video data
for the corresponding blocks?

I have thus developed a plan of attack where:
I test the original sector as indicated in the UDF File table.
I test the sectors at the end of the totle set where the bup is supposd to be.
I test the sectors prior to the next title set incase the table length is incorrect.
i test the sectors directly at the end of the final video packet...

if all the above fail... what next?
there is a possibility of more data between those regions...

[geezerbuttz]

ok...

I decided the best improvement was to attempt to read the data in one blob , then process it n memory for speed...
much faster .
Side effect you may allocate a large block of memory ... ( not happened yet , but it is possible)

if the data block cannot be read , the Bup is abandoned.

I def need to place some system in place when loading up for DVDSectorprobe , as the app just sits there while it rebuilds the structure for probe...

I will pop a window tied to the controller to display the loading info.

I ned to upgrade the probe bup handling , as it missis bups in sector compacted layout schemes...

Im going away ( possibly ) on Saturday , I will take a laptop with me though to do some work while Im away...
Not easy on a small laptop.

[rthpjm]

Hi All,

I have been developing a robust BUP reader to get into Probe and MTR.

On a normal disc , the BUP is located in the last corresponding sectors at the end of the title set.
That way , if the IFO fails , the player can easily located its backup.

On discs these days , we have to contend with ifos that have deliberate errors in them that invalidate the end of the titleset address,
Bups that are simply missing , or located in a non standard place.

While running tests on ISO is not as issue as the speed i good , locating by sector scanning on a genuine disk can be painfully slow.

I need some feedback on how to best handle this,.

Should we limit the check to looking to where the bup should be , perhaps extending the check to the end of the video data
for the corresponding blocks?

I have thus developed a plan of attack where:
I test the original sector as indicated in the UDF File table.
I test the sectors at the end of the totle set where the bup is supposd to be.
I test the sectors prior to the next title set incase the table length is incorrect.
i test the sectors directly at the end of the final video packet...

if all the above fail... what next?
there is a possibility of more data between those regions...

Hi GB,

I don’t know enough about the structures to offer definitive suggestions. However, I would suggest using the faster approaches first, at the point of needing to move to the much slower search, maybe offer a prompt to continue or “fail”, or an option in the app settings?

[rthpjm]

ok...

I decided the best improvement was to attempt to read the data in one blob , then process it n memory for speed...
much faster .
Side effect you may allocate a large block of memory ... ( not happened yet , but it is possible)

if the data block cannot be read , the Bup is abandoned.

I def need to place some system in place when loading up for DVDSectorprobe , as the app just sits there while it rebuilds the structure for probe...

I will pop a window tied to the controller to display the loading info.

I ned to upgrade the probe bup handling , as it missis bups in sector compacted layout schemes...

Im going away ( possibly ) on Saturday , I will take a laptop with me though to do some work while Im away...
Not easy on a small laptop.

I like your idea of using RAM allocation, slow read once (into memory), then fast read as many times as necessary.

Of course, I have a lot of memory in my rig, I’m all for it.

Again, maybe an app option to use RAM (default on), with advice to disable for low RAM users?

[geezerbuttz]

yes , im against giving the user to many choices , as i learned from the past..
"people dont like to think"

So , i set an upper RAM limit for the buffer size , and if is higher than that , we read in (n)CHUNKS of MAX_SIZE.

Right now the (n) prong approach is the way I have gone....

First if we have sectors from MTR , we verify that sector,
Second , we look at the original sector as indicated in the UDFFileTable,
Third, we see where the IFO says to look...
And forth , we find the next Sector with a valid IFO that is PAST the video data ( as they stack IFOs in the table we cannot simply grab the next ) and we look to see if we have an empty space between here and its previous node.
If the node is large enough to contain the expected backup , we start to read until we find the BUP.
Of course , we need a buffer of the original data to memcmp on , if we dont , we might match up with the wrong Bup...

And fifth:
We start to scan every empty area of the disk for the BUP header... Im not happy with this as it could lead to corrupted sectors being found , and slowing the whole process downn I may abandon this method.

[geezerbuttz]

right we are now in cosmetic stage and cleanup.

I wanted a progress indication when loading up the DVDSectorProbe.
As its has to build a disk map to pass , I needed some sort of UI to notify the user of the operation.

After a few different tryout , I decided to settle on adapting the DiskVolumes table , and presenting the process panel in the tables disk entry.

Its logical really..

Thats done.

next I want to improve the probe loader .
Currently it will always build a disk structure when you launch the probe , regardless if you already have a disk loaded for extraction.
That is overkill, and a waste of time... so I need to check if the disk is loaded , and reuse its device map if it exists.

Thats next...

Testsed the new structural rebuilder on a nasty that I have ,,,.. it passed with fling colors...

More soon.

[geezerbuttz]

ok.. that is the last fix for now.

What did I change:

I added a weak pointer in the disk class that holds a weak copy of the rebuilder structure.
That means that if you have a disk loaded in the extractor table , and you need to do Probe of that disk , it no longer has
to rebuild the complete structure from scratch.

Of course , if you do a probe first , you do not get the same advantage going the other way... it still needs to build the extractor structure.

It does help , as its quite a speed increase.

Been testing it now... it seems quite solid.

(BTW , i had an idea for a cheap and cheeky extractor based on the ISO extractor. We can use the ISO table to extract a basic disk in file mode , but , where they replicate sector blocks for the video TS.... do exactly what they do but use low level symlinks on the disk structure to mimic the original disk layout. You cannot burn or shrink these ( they replace the symlinks with hard links ) but it gets you a HD playable disk with cheap and cheerful playback to test ( and its not 50gb))

A fun extractor before RnR comes back.

More soon.

[geezerbuttz]

found that the update to probe reading code only worked via ISO .
Fixed for mainfeature.

Fixed a few Analyser issues regarding subtitles.

Running tests ..

At long last I feel I can do the video very soon ( its an amendment to the originals due to changes )

On a sad note , my brother-in-law has passed away suddenly..
Ive gotten sick in the chest with a nasty bug. ( not covid )

So , im a little absent minded right now.

We soldier on.

[savemulder]

my kind regard to you and your loved ones <3 , thank you for your tireless efforts

[geezerbuttz]

So , the world continues unabated...
As I was able to accelerate the disk rebuilder after the initial read , i was pondering to extend this to all methods..

Yes , its possible , but currently it looks a little 'hacky'

I intened over the next few days to change the rebuilder interface to better accommodate the needed functionality.

If home alone for the next 14days , so I can pretty much devote all working hours to this.
Its starting to look very good ( not that it did not to begin with ) and i promise , this is the last addition/ change i intend to make.

G

[TheSystemHasFailed]

my kind regard to you and your loved ones <3 , thank you for your tireless efforts

+1 - Family first GB. Hang in there sir!

[geezerbuttz]

Bulk loading has revealed an issue in MF mode regarding the validation of the title set count.
Thats fixed.
Identified and fixed a stack corruption when a valid IFO/BUP size cannot be generated , fixed.

Updating the main volume loader contextual menu layout to group function is sub menus.

Need to add enhancement to bulk remove to allow user to remove all extractors based on type , and not just "all"

Bug hunting , user happiness , thats all...

[geezerbuttz]

Addendum video is complete and uploaded.
lilAbmer can link it to the forum now if he so wishes....

had to drop the mkv ARM version as they are not static , and cannot be dispributed outside the current build.
I can at some time change that , but , I need to get other things completed here forst.
G

Oh , testing , check , optimizing etc.

Geting close.

[geezerbuttz]

ok the days go on and the long job continues..

Im building a library of IMF that I have genuine disks for.
This involves mapping , extracting , checking.

I have found one disk that prompted a small but logical extra check , and this in turn revealed a slight byte alignment issue.
Thats all fixed so I can continue.

I found that the Software update framework was reporting a network error when the core .plist was missing from the download location , so i expanded the return code types to cover that, this caused the silent operation mode to break , prompting the panel always on first launch , so that go fixed.

The updated framework is now in both mtr and sector probe.

Disk mapping continues.
G

[geezerbuttz]

Everything is going well.
A couple of crashes , but that seems to be related to the disk being bad , and having to force remove it.
If left , bad disks with hard errors , will error out during Sector probe.( dont force eject )

The longest I have had is a house of mouse disk , about a small mermaid.
2.5 hours to fully map.

I know that a long time , but if you have 1000 people all having to map a disk that takes 2.5 hours thats a lot of time.
But if you only have one person that maps it , 999 simply load the solution.

Thats nothing in the grand scheme of things.

If a solution is available , MTR will extract it. ( it can extract without a solution , but it will take much longer )

More to do , library is growing.
I hope once it gets released we generate a substantial library of solutions. Not just new disks , but all disks should be mapped.

The maps are useless without MTR as they are encrypted.( they contain no useable data to the average joe)

G

[geezerbuttz]

A slight change in DVDSectorprobe internal operation.

I found that when processing states , if a state is found to be invalid , or if its PGC is invalid , it would stop the whole process.

It should of abandoned this state , and moved to the next state , or exited to the next full node.
Thats done.

MTR ISO slicer seems to produce too many continuity errors when building its slices.
Im not happy with that , so I will devote a few hours going over why it produces overlaps in the sector layout.

While they do get corrected before final loading to the UI , im concerned that they are there to begin with ,as it indicates the algorithm is failing under certain conditions.

I will investigate before release , and improve it.
G