MTR 6 ANNOUNCEMENTS - READ FIRST!

[geezerbuttz]

OperationQueue is in place , basic code to test data passed in for extraction of Iso.
Seems ok , a few tweeks to do in the morning then to flesh out the ISO code.

Full dependency cycles on volumes with the same data/mount points , you can queue as many as you like and execute them in order.

Need to update the naming scheme to prevent overwriting files ( same as in Batch using UUID i guess )

Code is modern , clean , and above all far more optimised..

more soon.


It all has to be rewritten from the ground up as the old code is not suitable to be used.

[TheSystemHasFailed]

Hey ! we are getting the Iso data to the interface... super clean code!
Will attempt to optimise it more though.....
Found a strange issue though , and I guess its down to ARC.

When malling a chunk of memory , if the chunk is not cleared and 'initialised to zero' the compiler throws a serious fit.
I have been moving over to calloc , it takes a little longer but it zeros out the block , and keeps the compiler happy.

who knew....

Wow.. I'm an old C and Unix guy - and I even wrote a basic interpreter and an OS as part of my school work. I'm kinda surprised, but not...

The intent of Malloc was that the programmer would be responsible for memory management in what was called the "K & R" days. But you were not specifically expected to pre-clear or pre-initialize the memory - however if you did not.. you would likely find yourself in a nasty position with respect to core-dumps etc.

I find it odd and counter-intuitive that compilers would now specifically call out that you have not pre-initialized the memory - but given that is one of the single biggest exploit vectors used by hackers (i.e. try to malloc memory that isn't yours and then overrun and execute random code) I suppose I should not be.

[geezerbuttz]

extraction has happened.
Just block mode in ISO , but the UI is operational.

Im currently is the slice generator as it needs some work to deal with the dreaded 'read loop' that sometimes happens.
Im going away to visit family next week for 7 days , but will have laptop with me to do a little.

More soon

[geezerbuttz]

The slicer has been rewritten , hopefully combating the recursive extraction loop on ISO disk.. i have to do a lot of testing.

I have implemented the slice structure into an ArrayController , thus if anybody wants it the ISO panel can include a tableView listing the slices and the states of them ( start sec - end sec state , etc)

Any thoughts on that , or should I just put my head in the oven..

Seen that MainFeature can miss vital titles , giving the completely wrong choice for extraction ( on these clone sector disks )
That will be addressed.


Right , im going to rest , been glued here most of the day.

[TheSystemHasFailed]

I have implemented the slice structure into an ArrayController , thus if anybody wants it the ISO panel can include a tableView listing the slices and the states of them ( start sec - end sec state , etc)

Any thoughts on that , or should I just put my head in the oven..

I would love to see that kind of functionality... that sounds pretty amazing...

Any thoughts on that , or should I just put my head in the oven..

Please don't.. some days watching your craftsmanship on this is the highlight of my day...

[geezerbuttz]

That all done apart from a few early cancel issues i need to run down..
You can now save out the whole disk structure..

[geezerbuttz]

That interesting the update feed for MTR has been taken down....

Hmmm.....

[geezerbuttz]

All utility code for driving the new ISO extraction is complete.

Still have to add the navExtraction engine , that's no issue really.

have to fix the agressive title reduction in Mainfeature.

Intent to look at the basic Iso slice as it seems to group data in a slightly chaotic manner , could be improved.
Will be away as tomorrow for a week , but will have work with me for the quiet wet days ahead.

See you all soon!

[geezerbuttz]

Back in the fold:

Had some time to get a perspective on some things regarding the new image file data.
Decided to host them online... Going to have the app load up a plist at start from the server ( we can't list a directory ... we have to parse a file ) to build a list of remade/downlodable IMF images. this list will be in addition to IMF that you have local on the system.

You will be able to load a IFM from online this way without going to the forum.

IT requires a few changes , i.e we need to include the disk serial number in the IMF name , The online plist will need to be edited to reflect new entries ( im looking to automate that , but its not looking good )


also decided to add a new ISO mode that does not rip , but does a very low level ( sector level ) mapping of the disk to build a deep IMF file.
I will have to update the IMF internals to identify the two types....

More soon.

[geezerbuttz]

Big day..

Local on online system for IMF's is running.
A real pain is that we cannot scan ( securely ) the director structure of the IMF folder , thus we have to have an Index that has to be manually updated when a new IMF is added.

I could go the FTP rout , but then the security is out the window , as the FTP pass is broadcasted in the clear.
Not good...

I may make an app that camn mount the ftp folder , generate a new plist from the content , overwrite it etc...
That cannot be in the app as I do not want users having uncontrolled access to the secure location.

I updated the ISO slice indication , now providing a green tint background in the tableview for "void data" , red tint for "encountered bad data" and standard grey for normal.

Bad data is always in preference against void data , so adjusting the void data setting makes no difference in the internal bad data cell calculation. Once bad , always bad , until you reset the table.

Tomorrow I will spend more time on the Directory reading issue and seems if I can come up with some other idea... maybe we can make that folder as read only fro a web view , I can parse that data..

Comments , ideas are very welcome.

All that is left is the NAVTRACK extractor , a slight alteration in the slicer , the deep analysis reader , and a change in MF to stop it loosing Titlesets.

Getting there.....

[geezerbuttz]

Hmmm , a few more bugs out:

How would an option to automatically check the database to see if your disk is in there , and load it for you ?
Just a thought..... have to match disk serials for that to work I guess.

Or change the format online to hold name and serial as seperate entities in an array...
Just more complex doing that , so we can rebuild the table in code , as needed.. if we go down this route

[geezerbuttz]

Hello!

I finalised the format of the IMF data file and locked it down now..
It presents in the format of [ NAME - REGION - SERIAL NUMBER ]

The app presents the volume serial number for each extraction to help a match if there are several IMF's with different serials.

I completed the automatic loading of IMF;s from the network,
It only looks online , as only online can they be guaranteed to of been checked out by us.
If an IMF is 'manipulated' in a nefarious way , it simply will not validate and load into memory.

If you start to extract an ISO image ( yes , an ISO of an ISO , who knows..... ) the save functionality of the IMF system is disabled.

This goes for manual and automatic.
Why? simple , an ISO cannot give an indication of any structural damage that may be present , and therefore will not trigger any sector layout information data.

I need to fix a small issue with the save location not being respected , unless you reload a extraction , and then its on to the next stage...

Busy , busy , busy...
G

[geezerbuttz]

can we get a mirror of this thread so the users an see what is going on , it may generate some more action in this ghost town...

[geezerbuttz]

something to note here :

The IMF database on the ftp: is only read at startup of the app.
This is to keep access times down having data read from the ftp:

This does mean that if the database is updated while you're in a current session with app , you will not have access to the new entries until you restart the app.

I may amend this to read the data base on disc addition.. it will increase the data usage , but only by a fraction.

[geezerbuttz]

A little Extra:

While not rip related , I added some functionality to the IMF code:

In preferences you now have a 'pattern matching 'pop-up.
This lets you choose between { None , VolumeName , Serial Number , Strict }

None: lets you list the whole content of the online database , you will need to scroll to find your entry.
Volume name: Presents all entries that match your volume name ( region and serial are discarded )
SerialNumber: as above but with serial number ( discarding volumeName and Region )
Strict: An exact pattern match based on VolumeName , Region , and Serial number.

This should make it easy for you to find a match in the database if one exists.
Strict will only present the IMF that matches your volume:

Ok.. time to move on to deep scanning,,,

[geezerbuttz]

Update: decided to look at both locations for the IMF files on auto disk matching.

It was a bastard getting a async loading system to do things in a serial queue.
We strict filter , and stop on the first successful load , not just first match.

The notification system is disabled to make sure the Desktop is not filled with messages apart from the successful load , or a single failure message.

Its bed time , im done in for the day.
G

[geezerbuttz]

Update:

Completed the IMF Code ( again , a few small changes were implemented .)
Removed the autoload on mounted ISO , no need the ISO is patched and does not need to be mapped again.
You can choose to still load the map manually.

Fixed a stupid error that caused disks with multiple MenuPGC to blow chunks.
( Mr Memory strikes again!)

Found a a very stupid error is the original Utilities class ( now a framework ) that was returning the wrong VTS count... stupid, stupid , stupid! This was the reason titles were missing from the MainFeature , as we were not reading the full structure!
Fixed.....

Optimised all Async calls , and corrected a few RaceConditions that cropped up.

Updated core Iso ( BlockExtractor ) to be completely modular , core is now 25 lines of code long .. all calling relocatable methods.. Clean and easy to debug.

Tomorrow , I start the Deep Mapper , then finally the NavEngine...

Clean up , remove unused classes ... test, test , test , release!
Phew.

[geezerbuttz]

Navigation track engine is in place and working , needs to be cleaned up ,but its working.
I need to optimise , ( i.e apply some extra brain power ) the error sector probe routines.

Cell jump is the most efficient IMHO.

Exposed the DVDdrive RPC state and name , in code , and added it to the UI so you know what RPC your drive is.
Exposed the RPCScheme , but im unsure how to translate the value returned into Region Free v Region Locked.

A few more bugs crushed , Got the chance to run a few test on older , hard disks , seems ok.

I will start to build the IMF collection from my material I have here to start building the data base from tomorrow.
I may get pre releases to mods so they to can get a crash course on it , and to start adding to the library.

I need to code an app that can read the mounted FTP ( on the desktop ) generate an updated plist that you can manually FTP ( thanks to apple for read only FTP operations ) to the FTP site... should be fairly easy to do.

Night night all.

[geezerbuttz]

one for littleAbner : do you want the disk splitter built into the app?

[geezerbuttz]

So here we are again:

The new code for the DeepScan is in place , its rough , and needs a lot of TLC , but its working to a basic level..
Warning , its slow as we use EEC 16sector error correction to brute forceour way through.

I have some more tricks up the proverbial sleeve to try to get through it just a little bit quicker , but the whole point is , is for emergency , to allow you to get a deep map of the disc.

Its rest time , the wife has decided.

[LilAbner]

one for littleAbner : do you want the disk splitter built into the app?

Yes

[geezerbuttz]

one for littleAbner : do you want the disk splitter built into the app?

Yes

Good , thanks for the feedback.

[geezerbuttz]

Todays Update:

Scan routines have been augmented. Single sector probing , of corrupt sectors is very slow. A Disk took 8 hours to map this way.
I added a hybrid sector checker based on the SRI table pointers in the stream. It the table is available and valid , it will use this , and that can reduce the corrupted sectors down to 2 if the SRI table is valid. That dramatically reduces time.

Only issue are "void' areas on the disk , they hide a lot of garbage in them , and yes , as there is no SRI table , we have to step down to single sector correction reading == slow.

I will address this tomorrow, with a new idea I have.

I will also try to get the current engine to read nav_packs and not just single sectors , therefore stepping through the disk faster , but we still need to mindful that were are indeed trying to map sectors here , not nav_packs.

So , its a new hybrid engine , that is running...
Lots to clean up though , still have to gut a lot of older code loitering in the app.

Next Episode: "Geezerbuttz & the OAP home for derranged coders."

[mtr.admin]

can we get a mirror of this thread so the users an see what is going on , it may generate some more action in this ghost town...

All users who have MTR5 permissions are able to see this thread. Commenting has been disabled.

[geezerbuttz]

The Days Update:

Sick as a dog. pains in both armpits , its body flu.. good job I had my jab.

So , today , apart from moaning a great deal , I dismantled the DiskScanner.. A though occurred to me that goes as follows ....

The disk scanner structure is the same as the structure generated from the Ifo. This was a mistake , as if the Ifo structure is defective , and you nee to scan the disk at low level , basing of a defective structure is going to lead to a still defective result.

I restructured the raw data for it , grouped all relevant data , and we nibble reading poss data from Navigation packs , generating cell as we go.

Any tracks read , that generate an error , write out the good part of the read , followed by a small probing season to determine the end sector , then a dead cell is written , followed by fresh start to clear cell data.

I disable auto mapp saving for this method , meaning you must choose to save the map yourself.
Also , a reset is pulled on the base table before start , just in case you have a IMF already loaded , that will tied the results.

You can select void pack bypass , this will copy existing marked packs directly without attempting to read them.

Im away for three days on tuesday , so no reporting then.
Still have to integrate the disk splitter , but that's last on the list , asn is only a sub process anyway.
Need to clean up the older Navigation track engine etc , remove the last of the older ISO RnR code,,, test , test , test ..

[geezerbuttz]

Deep mapper is working , and grinds down to the nearest sectors , while maintaining a relative short time to gather the disk info.

There is no way to avoid surface issues when mapping a raw file structure , so , patience will be required.

I have implemented many tricks , but there are areas where we have to probe sector by sector..

I have a small addition to improve that. I will do the in the morning:

Other than that , It does provide a map of greater detail than a IFO based map...
So will be useful when the ifo structure is defeated.

More than one way to skin a cat.

[geezerbuttz]

Been testing a few other disks. Some are hybrid disks with HFS filesystem:

Adjusted disk Ident code to recognise them , but made sure we only allow disks with 2048 blocks per sector ( dvd compliant )

[mtr.admin]

A few long time users have requested that I unlock this thread for commenting/questions.

Done!!

[geezerbuttz]

found a disk , an older one , very old lol , that reveals issues in one part od the code.
That will need to be updated when I return from the Santa run.
G

[savemulder]

i can post ! thanks 2socks , and thank you GB !! i love reading your development , MTR6 hype-train !!!