MTR 6 ANNOUNCEMENTS - READ FIRST!

[geezerbuttz]

A quick look over and a step through the code , and Im thinking it will not be to much work to add some cache functions into the main IFO code to check the cache if we already have the IFO and return that , or load a new one and store.

That tomorrow.

[geezerbuttz]

well it took a couple of day , but its done.

I had a good look at the ifo reading code , ( there are two different types ) and consolidated them.

All different methods feed down to one concrete function that does all the heavy lifting.

I initially wrote a deep copy routine to give us unique backups of the cache data.
This proved problematic , and hard to maintain , esp as I re calculated the IFO data to correct for incorrect sizes on the file table.

This means the initial read , would have to be updated to correct it.( a complete rebuild )

I decided that the best way to do this is to cache raw data from the disk , provide functions to enable us to destroy and update the buffer if the ifo size needs to be updated.

when we need an ifo , we first check if we have a buffer in the cache, then compare the sector sizes ( if its == or the range fits inside the current buffer , we return the cached data , other wise , invalidate the cache , and reload and update the cache.

All in all , it seems to work well , and dramatically reduces disk reads,
I will take a couple of days to run it thoroughly to test it for bugs.....

The ifo cache is only destroyed on disk removal from the app stack.

G

[geezerbuttz]

popped out 6004 today.
A bug fix for the bulk loading mechanism , and an update for the 64bit array handling.


For 6005...

Spent the remains of the day gutting the core , to add the frameworks in a better manner. This ensures that all build fresh every time we build the core app.

As they are only referenced in the core , any fixes that a framework gets is passed up the line to any apps that share it.

A nice side effect is 2MB of the app at build time.

The VideoEncoder framework is complete , I will start to wire it into MTR next week.
I do have one niggle though , the quant function serves no purpose with MP4 output , so I need to code around it if its selected and the output is MKV with MP4.

More soon.

[geezerbuttz]

well its is encoding x264 - 265 10 bit.

Output is wrapped into mkv.

Not happy with the centralised controller applying global settings to all. Really needs to be a 'per stream' setting.

Thats for the morning.

I intend to add the color panel corrections into the preview code ,so you can dial in your corrections before going into the encode and setting the there.
A little visual feedback. ( most likely make it some sort of copy/paste system )
G

[geezerbuttz]

been a long uphill struggle .

Never easy when you add something you did not design for in the first place.

Anyways, the encoder is in , i have been debugging and optimising to get a decent output from the videotoolbox.
This now seems to be working well.

I had a slight colour shift that was a pain to hunt down , bit we got there.

Also , had a edge clamping bug in the NLMeans denoiser , it was causing a randon black blocks on the edge . thats done , and as a bonus , I have directed the Chroma through a multiframe threshold denoiser to smooth the chroma a little after the Luma is denoised , in improved image stability.

Later I will see about wiring in FFMpeg ( external binary , piping to it to avoid issues with GPL ) to get the better quality , albeit slower 264 /265 encode.

That for a bit further down the road.

Implemented coreImage cropping , and single generation colour and shapen filters.
You have 2 pages of filters ( Core Image )and filters that require more than one image in the buffer to generate the result.
You will see the layout in the encoder panel that becomes available when you select MKV output.

More soon , I need to run this hard to check output for validity.
G

[geezerbuttz]

Just tested the denoiser.

DVD: Noisy transfer of Forrest Gump.
( its a very bad transfer , extremely noisey )

600Mb Sample.

Normal at CFG of 65 486Mb.
Denoise @ CFG 65 - 285Mb.

That works.

[geezerbuttz]

right , i have been updating the Disk Builder ( burner ) as it needed some TLC to be clearer.

Its alot more verbose and a little less arcane now.

I have imposed disk validation on all possible formats ( make sure you can only add the correct info to a particular file structure type).

I have also added the validation stage to the DVD and ISO builder stage the rebuilds your disk format into a 100% compatible structure before burning. ( its a temp file that is held in the temp storage the OS provides , and is deleted when complete )

( i have to add it to the DVD stage in the morning. ).

It is currently 100% built into MTR , and Im thinking to move it to a framework. It makes sense as it can be the used in other projects.

Lastly , I need to produce a new usage video for the builder for you good users.
That will be added to the current user guides built in to the app.

And then , the hard project: I UDF/ISO19660 File table editor....
You can guess what that is for....

[geezerbuttz]

Experiment day:

I just replicated my first disk with hardlinks in it.
This is the process they use to bloat a disk upto 50Gb etc. They hardlink files to the same secors thus reusing them.

When you try to copythem normally the system breaks the link , creating a new , unique file , thus bloating the new folder / file.

Even though you have hardlinks , the finder still reads the total of the folder NOT resoving the hardlinks.

I have created copy routines , to maintain hardlinks, ISO code to create ISO with hardlinks , and routines to burn them to disk , maintaining the structure.

Im just looking into potability methods within the OS via terminal to move them.
Looks like TAR archives are best as they support hardlinks , and for direct copy , DITTO , as it supports hardlinks.

This is very promising , because it solves one of the biggest issues with full disk requant.

[geezerbuttz]

I need to do a little bit more work on the DiskBurner , as it is not taking into account the hardlinks on the file structure , leading to a bloated file size.

Fix that tomorrow.
G

[geezerbuttz]

fighting a very old , unmaintained , and partially broken DiscRecording framework.
It really is quite a pain to work with.

I may break away and let mkisofs build all the images , and just use the burn engine...
The issue is hardlinks , and symlinks ...

Hardlinks are broken in the framework , symlinks work , but only on HFS+.

I though of referencing the DRFile , and using the same reference in multiple places , hoping its smart enough to share the same root blob..
No , it optimises the references out completely.

Its a bloody pain.

[geezerbuttz]

ok , bit of fun here . I was never really aware that when i read the contents of a folder into a NSArray ( be it paths or URL ) the order of the files in a folder is not reflected in the array!

So for DVD folder ( VIDEO_TS ) im sorting into standard folder order.
For normal folder ( containing arbitrary data ) im going to sort alphabetically. you can drag reorder as you like.

The hardlink saga continues...

Right now we stand like this .

ISO9660 ( rockridge ) All duplicates and Hardlinks are Converted or Copied.
Joliet (rockridge ) All duplicates and Hardlinks are Converted or Copied.
UDF All duplicates and Hardlinks are Converted or Copied.
HFS+ All duplicates and Hardlinks are converted to symlinks
( if MP3 Disk , linking is disabled , all files are duped with increasing index added to filename )

Thats all working , with all atomic renaming and size and hardlink identification working.

Now to do the alphabetical sorting.


I still have to do the DVD folder purification prior to burn.. not to much work left.

[geezerbuttz]

i need to make some performance updates tomorrow.

Right now to use mkisofs to get the image , i create a clean temp file structure in the temp directory of the mac.

If files are big , this can be a little slow , not to mention the write overheads.
I create this directory to maintain a conformance to a specific standard before burn.

Im going to create this strructure using hard links from now.
They can be resolved correctly by mkisofs , so it will speed up the creation , and reduce the bloat.
Cu.

[geezerbuttz]

ok , that is all done:

I need to run some tests over the next few days to check it all out as there are a large number of changes.
G

[geezerbuttz]

Disk Builder:
Renamed the Disk Builder to "Build And Burn" Easier to guess what it does that way.
Added a directory manipulator to flatten the display hierarchy for MP3 player than only like one layer of navigation. ( this only flattens the virtual directory tree , not your local content)

Made the directory scanner silently bypass files it cannot read , rather than reject the who directory.

Main App:
Standardised the who app to use Apple certified Symbols. Thus maintaining a standard look across the app.

Misc:
I need to create two new user guides tomorrow.
One for the video (mp4) encoder addition:
One for the BildAndBurn ( formerly Disk Builder ).

Then weekend to test again ( never test enough lol )
Then its out.

Once that happens I will move the BildAndBurn to its own framework for future portability , and to allow others to wire it into their own apps.

[geezerbuttz]

Been thinking:

RnR mode is dead currently , im in thought how to go ahead in that direction.

We are already mapping a disk playable sectors in SectorProbe:
What if we also store its navigation chain?
i.e from firstplay to -> ? what menu goes where ? what happens when a stream end playback?
if we do not map titles without any video playback , but only map navigation to and from playable streams, we can streamline the navigation.

If we capture all that , there is no reason why all the VM Commands cant be stripped , and new simple direct navigation replaced , thus bypassing all of the nonsense we see on a lot of disks.

This could be expanded to give the use a choice how to change the navigation on a disk ,thus bypassing certain features if they so desire.

I need to look into this ..
G

[geezerbuttz]

Back home , fixing a bug in the transacoder for NTSC ..
framerate is borked.

[geezerbuttz]

ok , had to pre process data to get the correct pulldown flags to compute the correct Film framerate for NTSC.
Done.

It has to read a few blocks before we start encoding.
Currently we do not switch framerates when encoding ... some crazy disks do have 29.98 for credits and 23.976 for the main. , go figure.

I was able to move some recursive calculations to one shot type , in the pre read, reducing overhead on the main encoder.
Moved to CMTime to correct frame timing .. ( silly boy )

Frame rates are now solid and stable. Sync is maintained.......

A few more things to look at , then wrap up.
Rushing as I need to leave again on the 16th.

[geezerbuttz]

running some encodes. All seems ok , multi encode , with different framerate disks , and regions ( PAL - NTSC ) seems ok.

Im just waiting for a problematic IMF to arrive to I can see why it blocking the UI , or not registering on the table.

Patched an omission , where I forgot to set the aspect correctly on encodes where the preview and crop was not set.
G

[TheSystemHasFailed]

Back home , fixing a bug in the transacoder for NTSC ..
framerate is borked.

Thank you for working on the fix..

[geezerbuttz]

ok , release is in the wild:

I updated the IMF handler to verify the file name is of the correct format.
Seems that hacked files that did not match the structure needed for the reader were causing a failure in the population routine.

Thats now handled , and any IMF. DVDSS etc that fail the standard check are simply ignored.

Dont forget to run 6005 and update.
G