> Home Assistant already knows which bins (trash cans, for the international readers) are due for collection. [...] What it can’t tell me is whether I’ve actually done anything about it.
My first gut reaction to this was: Have you tried using a vision-language model?
I used to have an elaborate Home Assistant setup with tons of zigbee sensors here and there telling me if front door is not closed properly, if the rice cooker is left on (via current monitoring), if the garage door is left open (with a magnetic contact sensor), if window is left open, if the soil is dry, etc. I had to keep changing batteries of all these things and manage flaky Zigbee connections in the past.
More recently, I've found the absolute most reliable, ~zero-maintainence solution to be an LLM. Home Assistant already has an LLM plugin, you can wire it up to a local LLM if you want, and you can just feed it an image and ask "Is this garage door open or closed" and I found that to be much better for my sanity than changing batteries every 6 months and dealing with flaky sensors sending me false alerts.
I even found it useful for "Did the cat vomit on the floor" (if so notify me and pause the cleaning robot schedule) and "Do any plants look like they need attention"
For TFA's use case, if you already have a security camera facing front, you can just feed it that image and ask "Are the garbage bins out"
Also, if you're worried about cameras sending home images to the cloud, there are non-cloud options. (I use Unifi cameras and Home Assistant gets a RTSP stream over LAN.)
> Or the bin’s precise location - how many fixed reference points would I need to triangulate it on a 2D plane?
Back in 2019 I worked for a company (in San Diego) that added UWB-based positioning to the 7 line in NYC.
The code I wrote ran on what was basically a combined Raspberry Pi 3 compute module connected to a UWB device installed into the train's bonnet which would ping UWB devices set along the track side to determine the position of the train.
In most situations I could get a unique position based on only two beacons, but I had the benefit of the trains being on a fixed track which severely limited where they could practically be in space... well, assuming there hadn't been a train derailment.
The system didn't work in a 2D plane, though, all of our surveyed track data and track side UWB positions were all 3D encoded as ECEF coordinates.
In some situations depending upon track layout you needed more than 2 to determine the correct position if you were just sampling single points in isolation, but since you're working with a continuous stream of changed positions as the train ran along the track at predictable speeds, you could pretty easily figure out which of multiple possibilities it had to be based on what came before even if you had potential multi-positional results for the current sample.
Of course, we would use as many of beacons as we could see to strengthen confidence in the positioning, and the beacons were placed so we could generally see a good 4-6 of them most of the time.
Snow was the biggest challenge for the system, in a heavy thick snow the UWB signals bounce all over the place and you get severe multipath effects that can make it pretty difficult to determine which is the true line of sight signal for your time of flight calculations.
That’s all well and good until the truck driver operating the mechanical arm drops your bin into the truck. Spoiler alert: the bin will not be fished out, it is now garbage.
So those tags are going to have to be a little cheaper before I’ll use them. :-)
Not the whole bin, but the operator managed to scrape off the hinged lid and both wheels from my recycling bin a few weeks back. Sent them photos and they sent me a new one.
Thanks. I regret not taking more pictures during the initial testing indoors. I should document these things better, I only think about writing them up when it’s too late
My first gut reaction to this was: Have you tried using a vision-language model?
I used to have an elaborate Home Assistant setup with tons of zigbee sensors here and there telling me if front door is not closed properly, if the rice cooker is left on (via current monitoring), if the garage door is left open (with a magnetic contact sensor), if window is left open, if the soil is dry, etc. I had to keep changing batteries of all these things and manage flaky Zigbee connections in the past.
More recently, I've found the absolute most reliable, ~zero-maintainence solution to be an LLM. Home Assistant already has an LLM plugin, you can wire it up to a local LLM if you want, and you can just feed it an image and ask "Is this garage door open or closed" and I found that to be much better for my sanity than changing batteries every 6 months and dealing with flaky sensors sending me false alerts.
I even found it useful for "Did the cat vomit on the floor" (if so notify me and pause the cleaning robot schedule) and "Do any plants look like they need attention"
For TFA's use case, if you already have a security camera facing front, you can just feed it that image and ask "Are the garbage bins out"
Also, if you're worried about cameras sending home images to the cloud, there are non-cloud options. (I use Unifi cameras and Home Assistant gets a RTSP stream over LAN.)
I’ve not played with vision models much. I did some experiments with OpenCV and Processing many years ago, but times have changed.
I think you just wrote me next weekend project ;)
Back in 2019 I worked for a company (in San Diego) that added UWB-based positioning to the 7 line in NYC.
The code I wrote ran on what was basically a combined Raspberry Pi 3 compute module connected to a UWB device installed into the train's bonnet which would ping UWB devices set along the track side to determine the position of the train.
In most situations I could get a unique position based on only two beacons, but I had the benefit of the trains being on a fixed track which severely limited where they could practically be in space... well, assuming there hadn't been a train derailment.
The system didn't work in a 2D plane, though, all of our surveyed track data and track side UWB positions were all 3D encoded as ECEF coordinates.
In some situations depending upon track layout you needed more than 2 to determine the correct position if you were just sampling single points in isolation, but since you're working with a continuous stream of changed positions as the train ran along the track at predictable speeds, you could pretty easily figure out which of multiple possibilities it had to be based on what came before even if you had potential multi-positional results for the current sample.
Of course, we would use as many of beacons as we could see to strengthen confidence in the positioning, and the beacons were placed so we could generally see a good 4-6 of them most of the time.
Snow was the biggest challenge for the system, in a heavy thick snow the UWB signals bounce all over the place and you get severe multipath effects that can make it pretty difficult to determine which is the true line of sight signal for your time of flight calculations.
So those tags are going to have to be a little cheaper before I’ll use them. :-)
Luckily these are only £5 each https://www.alibaba.com/x/1lBJtin?ck=pdp
The majority of cost is the wrover and the programming jig (which I think the manufacturer built to order, it’s very… home brew