Does Your Instrument Strand Its Data in an Proprietary Binary Format?

This week we hosted the first REAL (Readiness & Engagement For Automated Labs) Summit; today that was followed by Ginkgo Datapoints Summit.  Both were fascinating surveys of the leading edge of automated data generation and feeding that data into AI models.  That data generation relies on linking sample preparation instruments to data generation instruments.  Two of my recent pieces were around issues of interfacing with instruments: Anthropic's proposed but not disclosed Model Hardware Standard and some of my (perhaps painfully naive) stabs at evaluating instrument Application Programming Interfaces (APIs).  I had commented that some instruments lack APIs - but what if even worse the data is still locked into a proprietary format?

Electrophoresis is a venerable technique in biology and an early one I ran on a regular basis.  For Ginkgo Cloud Lab, we have workflows that use the Revvity LabChip capillary electrophoresis to profile proteins expressed in our Cell Free Protein Synthesis system, either before or after purification.  The LabChip results give important information beyond just quantity.  For example, if you see your protein at the wrong size then perhaps it has been chewed up by one of the residual proteases.  

LabChip was developed by Caliper about three decades ago; it's a solid, mature technology.  Which is what companies like Revvity like, mature products that require little to no R&D expense and can simply be fed into their extensive sales & marketing network.  

My gripe with LabChip is the data - it's locked into a proprietary binary format.  That was once a recipe for locking customers into expensive licenses for software tools to read and manipulate the data.  But Revvity now gives away the software - it was a free download!  Windows only I think.  You can export some of the data from tool - generate CSVs of peaks for example.  But it doesn't appear possible to extract the full signal data from the file as numbers - just as PNG image files.

Which means LabChip is gong to be locked out from any sort of large scale AI training, unless someone cracks the format.  Now I'm not certain anyone wants to train at scale on electropherograms, though these days isn't the rule train on everything?

I hadn't run into this issue for a while, but that's largely since I was a FASTQ/BAM fiend. NGS never really had the problem - open formats took over very quickly.  ABI used to pull this stunt with their AB1 format for Sanger data, and was known to break hacks of their files when they changed the format, but that was abandoned decades ago.

Which meant open source libraries to parse these files were widely available.  Google search says there is no such libraries exist for LabChip ".GXD" files.   How can this be?

It might be possible to hack the format - it would be an interesting college student project.  Maybe an AI trained on a huge number of files with no context could figure out key structural information.  Better would be to export a bunch of electropherogram PNGs to train on.  Electropherograms as a cryptographic problem - a great intellectual challenge but a bit ridiculous to have to do this?

The simple solution would be for Revvity to just post the source code for their software on GitHub.  Since they are no longer charging for it, what's the risk?  It could certainly accrue some good vibes.  Better, it would keep their instrument relevant in a data hungry, AI-driven world.  If using your instrument and analyzing the data can't be fully automated, you're a horse-drawn buggy supplier in a world of autonomous automobiles.