Hardware interface

[FFY00]

Hi,

Would it be in the project's scope to add a way to interface with capturing hardware?
If so, I would be happy to help contribute that feature.

Cheers,
Filipe

[wcjohns]

Hi Filipe,

I wouldn't be opposed to accommodating this.

But, I think any implementations for specific hardware would have to be a fork. The mainline could perhaps have the generic mechanism for adding the acquisition interface/logic, probably behind a compile-time option, and then any forks could implement the hardware they want.

However, perhaps the better way to go may to instead embed InterSpec inside of other applications that perform acquisition.
That is, for some specific hardware, you create your acquisition logic and GUI in whatever language and frameworks you want, using the natural logic-flow for that hardware. Then, since InterSpec is rendered in pure HTML/JS, you can just create a window or frame with a WebView that is rendering (possibly a slightly modified version of) InterSpec - and pass off the acquired data to this instance for analysis. There are already some mechanisms in the InterSpec code for doing this (ex., see target/sharedlib - although I haven't tried this example for a few years, there could be some bit-rot I would be happy to fix), and we could likely easily add/modify the interfaces for this. It's been some years, but I have done this a few a few times, and thought it was a better fit than shoe-horning other programming languages and event loops inside of InterSpecs.

The other thing I have done a few times, and was happy with, was using components of InterSpec to create specific acquisition apps. Namely re-using the d3.js-based spectrum plotting code, and SpecUtils for reading/writing spectrum files and energy calibration and stuff.

Always happy to accommodate code re-use when we reasonably can.

Thanks for asking, and best regards,
-will

[PlasmaHH]

Hi, could this maybe also work the other way round, that is, have a plugin system that it just asks the data from? And maybe one default plugin that gets the data via network in a specific protocol (that you could define, maybe some json format?) ... I think that would be much less effort for people to get basic hardware support, especially for devices that are just connected via some serial device or tcp/ip one could easily write a tiny translation layer, rather than having to learn how to do a gui and whatnot.

Also this would make it far easier to connect multiple different devices and work with data from them at the same time.

[ckuethe]

I thought a little about this when I was adding Radiacode support and eventually came to the conclusion that the easiest approach was to:

1. write a stand-alone tool to convert existing Radiacode files to something InterSpec could read (a reasonable approximation of N42)
2. using what I'd learned, add a new parser to SpecUtils so that InterSpec could read Radiacode files without an additional conversion

... which is about 50% of the plugin system you're suggesting, albeit using N42 as the wire format.

I decided to stay far away from any hardware manipulation, since I couldn't think of any aspect of my Radiacode that I'd want to control within Interspec, I didn't imagine being able to do it better than existing radiacode utilities, I didn't want to try do cross platform hardware access, and I couldn't find any existing implementations in the code whose style I should follow.

[PlasmaHH]

As for the aspects you want to control, the first that comes to my mind is resetting the spectrum accumulation.

It is not about being better than the radiacode utilities but to be abel to integrate everything into a smooth workflow. Imagine you attach the radiacode into some shielded box for some experimentation setup and then you need some remote way to operate it and going back and forth between the app on the phone and interspec would be annoying.

[ckuethe]

I built a data logger using the excellent cdump/radiacode library and among the things I might do is reset the total dose or spectrum over USB, as I'm polling data. I've found it most useful to just write the captured spectra to disk, along with geotags, and separately reformat this stream of mixed up geotags, count rates, dose rates, and channel counts into whatever I need: rctrk, rcspg, gpx, kml, N42...

As I don't have access to any other interesting detectors, I couldn't think of a good way to add tighter hardware integration for any of the other detectors that SpecUtils supports. Anyway, this wasn't meant to say "don't try it"; only that I chose the easiest way for which was just to support reading data files I have the ability to generate.

[wcjohns]

The plugin system that primarily asks for the data is better than what I was thinking.

I still feel a bit mixed if there is a lot to be gained by integrating the acquisition into InterSpec, rather than a nice handoff, or someway integrating InterSpec into the acquisition program. But I am not a particularly good judge of stuff like this (either the UX or work involved), and everyones workflows are different, so I am totally happy to do reasonable things to support related efforts, as long as they don't detract from the nominal file-based workflow if InterSpec.

For the mean time, is there maybe a way we can make getting the data from phone, or other desktop, apps into InterSpec easier/smoother?
For example, we defined a URI-scheme and QR-code spec (see here) to help make it easier to transfer spectra from a detector (with a screen) to your PC from the field (you can just drag-and-drop a picture file of the QR code onto InterSpec, and it will open as a spectrum), or to hand the URI off between programs using the standard OS mechanisms. I am not a Android/Windows user, but if there is something we can do to make things smoother/easier, I'm happy to.

(and sorry for delays in replying - I have been off work for the last little while, and will be off through this week)

[PlasmaHH]

Don't worry about the timings, we get what we pay for ;)

I know how it can be difficult on the UX front, especially if you are intimately familiar with all the things that go on, it is easy to overlook which things may be less obvious to the beginners or people that just want to deal with one problem.

I am still struggling a bit with the program overall for my hobby needs, but am still also far away from suggesting something that would make it easier for me. Also I am not much of a windows user, rather linux, but I think the principles should apply in the categories desktop vs. mobile.

To the question at hand: the QR code "standard" looks quite intresting and it would be nice if soemthing like that could be adopted. Even though maybe not totally on-topic for this issue, allow me to drop an idea on a tangent. I did not went through with this but for similar kinds of data in the past before that deflate step, I used some differential stopbit encoding with success. I have not yet tried to have a look if that would save significant amounts of data, but the displaying program could also try out multiple versions and display the smallest.

By differential stopbit encoding I mean this: Calculate the difference between the current and previous step (the following is using 8 bit bytes but can as well work with any size, for certain raw sensor light spectra I had that encoding go with 10 bit much better). Take the sign as one bit (0x80 or not). Then if the value is less than 64 set bit 0x40 and encode the rest. If it is bigger, don't set 0x40, encode the upper 6 bits and proceed to the next byte, but there do the same with 0x80 as the stopbit and repeat until done. This is a bit of encoding/decoding work but for smooth functions with high values this can save a lot of data. One can even do more shenanigans with using 0x80 as stopbit always and encode the signs in a different "bit array".

Now back to the question... There are surely more formats out there than devices. Not for all devices it would be possible to implement anything in them to ease the data transfer, or would be feasible. Let me think out loudly here a bit:

• devices like radiacode that on its own have only BLE access (or really any other) with proprietary protocol -> This would require some kind of plugin
• Devices with a display that can be controlled -> this could use the QRCode mechanism.
• Devices that run with apps: It is not really that different to save a file from within the app than to make a screenshot with a qr code -> Fileformat should be supported

So the real thing that is missing here from being smooth is how to get the data from the phone or whatever into interspec and one thing I was wondering is, how sensitive that data is. Thinking more loudly, I could imagine there being a web service, similar to a pastebin or so, and the QR code being a URL that fills a new entry. That entry could get some ID that is not searchable by the public (but visible as long as you have that ID). This could be entered into InterSpec easily and it would then download the spectrum without havin to copy and files around... That service could also maybe accept all kinds of other formats or files that interspec will then be able to interpret...

one would just have to make sure it will not be a new filesharing service ;)

[ckuethe]

Perhaps I should try learn something about android dev. I'd really like it if I could share a spectrum/spectrogram/track from the radiacode app directly to interspec. This seems like it should work, because other apps can share to interspec... currently I have to share to google drive/nextcloud, and then import to interspec.

[wcjohns]

As far as sharing from radiacode app to InterSpec on Android, maybe this is just an intent thing - like maybe there is some way we can just put in the InterSpec app manifest that it accepts Radiacode files, similar to how it does for N42, SPC, CSV, etc.?
Building for Android is very inconvenient for me, and I'm not an Android user, and I don't have any other contact with developing for Android, so there is a lot I don't know. But I have added this to my TODO list to check out (but honestly, I don't even know how to test what the Radiacode app uses as mime type, or whatever).

differential stopbit encoding

I did actually try a a few of these types of things - expecting them to work better than the Stream VByte that ended up working the best, on a rather large corpus of test spectra - I was surprised.

Devices that run with apps

It would be nice if these apps supported the raddata:// URI mechanism. With this, its really easy, on all the operating systems, to just hand the URI off to the OS, and then it will take care of delivering it to InterSpec, or whatever other program wants to support the scheme - you can also just share the URI with other devices through the built in OS capabilities as well - this eliminates having to bother with the QR-code as the middleman. And you can also embed the URI in html links, so that if the user clicks on the link with the URI in the browser, it will open the spectrum in InterSpec (which is convenient for sharing spectra online, without having to use a file sharing service - although I've only tested with plain HTML sites, not any social media sites).

[ckuethe]

Building for Android is very inconvenient for me, and I'm not an Android user, and I don't have any other contact with developing for Android, so there is a lot I don't know. But I have added this to my TODO list to check out (but honestly, I don't even know how to test what the Radiacode app uses as mime type, or whatever).

I'll slap a debugger on my phone and see if I can get that info. I am very much an Android user and would like to figure out a way to build test versions myself, like I can with Linux desktop versions.

[wcjohns]

Thanks Chris!

Btw, if you do build for Android from source, I spent some time making the app a lot more-usuable on phones (rotate landscape for spectrum-only view, like before, but rotate to portrait to have all the tabs with useful tools; I also removed full-screen mode, which I think you had recommended, to not mess up switching apps), but hit a block that the soft-keyboard may cover up input elements, so don't be surprised by that...