Allow multiple cameras per printer for AI-failure detection#637

A way to add a web cam stream from another camera to the AI detection - I’m thinking about a camera looking straight at the nozzle to look for blobbing on the nozzle.

I’m afraid our AI detection is not a “first layer/nozzle cam”-optimized model, so that currently wouldn’t help much. Will let the suggestion stay and act as a “such an AI model would be neat” suggestion I guess! :) Although I can tell you that this is unlikely to happen at the moment - but hey, let the votes change our minds! :D

I don’t know enough to add much but I would have thought that if we know the printer is supposed to be on layer X and there is nothing on the build plate, the AI ought to be able to detect that the build plate is empty and at least suggest to look at the printer to see if there’s an issue, even just using the image from the timelapse cam? Thanks for the quick response by the way!

For sure! A lot of that works in theory, and we did discuss that many times with our AI partners. But it’s apparently more complex that one might think, as it’s suddenly no longer “detect something that looks like spaghetti or warping”, but “detect.. air…” - that means it needs to identify the bed, the nozzle and tool head, and the gap between the two. It’s not an impossible task, but it’s a different kind of model, and knowing how complex and big the test model we had going for object detection on the bed was, it’s another league entirely.

What we could do is pair our detection AI up with a vision AI, like Gemini or ChatGPT via their APIs: we give them an image and simply ask “is it extruding nothing?” (Or a set of questions). It then, without being able to draw boxes on it, calculate anything and so on, tries to “see” if it spots this (it essentially tries to describe the picture). This could be done, and it’s what we did with our “Is the bed properly attached and is there something on it?” AI detection for our AutoPrint feature, with varying success. Though with the advances in AI, both in capabilities and in affordability (as we’d pay someone else, Google or OpenAI for example per image analyzed), so it’s not completely off the table. The last concern here is privacy; we’d offload this to big, often American, tech giants, and give them pictures of your printers. As a EU company that is very privacy minded, we try to limit the amount of third party vendors - especially the amount that deals with your data, and especially non-EU ones (due to “GDPR”).

That’s the long reply!
TLDR; our current model wouldn’t be able to effectively do this, but from a technical standpoint it’s not impossible - just have to weigh pros and cons.

Extra note; not to mention that we enter object tracking and the AI having to know how the print should look, on a moving bed, from various angles; essentially we’d have to do some sort of mixed reality to be able to place the digital 3D model rendering on the bed at any position, for beds moving back and forth, up and down, from the top, from the front, from the side and at an angle. It’s a logistical nightmare.

I’m sure, however, that we’ll look back at these challenges in a few years and have modern AI-powered supertools to make this a trivial problem heh. But yeah, we’ve thought about it, checked it out, talked to the experts and the verdict is; probably not worth it at the moment.

My thinking over the past few years has also been that much of this should be fixed by the printer - by sensors. And to a degree we are moving in that direction. That also means that our efforts towards more advanced AI failure detection may be rendered obsolete in a few years, with all but legacy hardware. That’s part of the risk too.

Sorry for the essays! It’s a subject I’ve thought a lot about, and sharing our thoughts publicly helps the next user with this question to have it answered in full! (and, we believe in transparent development, being real and honest with our users; none of that “great idea! We’ll check it out!” if we know it’s never gonna happen…)