I'm setting up a household water-tank level sensor that I'd like to be able to monitor from (practically) anywhere to a mobile device or remote PCusing the above-mentioned ESP8266-based setup. The device is sampling at 5min intervals, each sample is a
single integer range 0 - 230 (cm of water height). Also toggling a 'heartbeat' boolean variable to indicate it's still alive.

Just getting stated with the ESP8266 and Arduino IoT Cloud via WiFi which was easy & convenient as it only took a few hours to get a solution up & running...

However, the free Arduino IoT cloud is limited to 1 day of historical data (I'd like months to a year). It also hides the bulk of the cloud API (which is convenient) though I'm not sure yet how to handle recovering from WiFi outages etc.

Internet searches uncover a ton of options many of which are far to up-market for my purpose.

It also appears there's another route using a local server (eg RPi) that receives the data (via NodeRed or similar) but then to access outside the local WiFi network means exposing a web server or similar - sounds like an extra tier or development and
complication/maintenance.

Any advice would be helpful...

--
Cheers,
Chris.

--- SoupGate-Win32 v1.05
* Origin: fsxNet Usenet Gateway (21:1/5)

Il 03/02/2022 03:38, Chris ha scritto:

I'm setting up a household water-tank level sensor that I'd like to be able to monitor from (practically) anywhere to a mobile device or remote PCusing the above-mentioned ESP8266-based setup. The device is sampling at 5min intervals, each sample is a

single integer range 0 - 230 (cm of water height). Also toggling a 'heartbeat' boolean variable to indicate it's still alive.

Just getting stated with the ESP8266 and Arduino IoT Cloud via WiFi which was easy & convenient as it only took a few hours to get a solution up & running...

However, the free Arduino IoT cloud is limited to 1 day of historical data (I'd like months to a year). It also hides the bulk of the cloud API (which is convenient) though I'm not sure yet how to handle recovering from WiFi outages etc.

Internet searches uncover a ton of options many of which are far to up-market for my purpose.

It also appears there's another route using a local server (eg RPi) that receives the data (via NodeRed or similar) but then to access outside the local WiFi network means exposing a web server or similar - sounds like an extra tier or development and

complication/maintenance.

Any advice would be helpful...

This is a big issue for our poor embedded developers. We are used to
create "low-level" software that are near the hardware, but now the big
part of the software is composed by "high-level": software that runs on
24h 7/7 connected servers (backend) and software that runs on powerful
usually connected device, mostly mobile (smartphones, tables, desktop PCs).

It's difficult to have in a single person all the skills needed to
create a global IoT system, from the embedded devices to mobile
frontends, passing by backend services.

Moreover, if 90% of embedded systems are similar (for example, most of
them run software written in the C language), high-level software can be created with a multitude of technologies and commercial/free services.

If this isn't a commercial consumer product, I think I will end up with
your last solution: a local web server exposed to the Internet. A stanrd ubiquitous web browser running on a desktop or mobile device can be used
as the frontend.

If you are creating a commercial product, you can't use a local
web-server, you need a Cloud system and a modern branded mobile app.
There are many technologies for this kind of systems, both for Cloud
(backend) and for mobile apps.

Anyway I don't think you can find all these skills in a single person,
mainly if you want to create a reliable system.

--- SoupGate-Win32 v1.05
* Origin: fsxNet Usenet Gateway (21:1/5)

On 2022-02-03 Chris wrote in comp.arch.embedded:

I'm setting up a household water-tank level sensor that I'd like to be
able to monitor from (practically) anywhere to a mobile device or remote PCusing the above-mentioned ESP8266-based setup. The device is sampling
at 5min intervals, each sample is a single integer range 0 - 230 (cm of
water height). Also toggling a 'heartbeat' boolean variable to indicate
it's still alive.

Just getting stated with the ESP8266 and Arduino IoT Cloud via WiFi
which was easy & convenient as it only took a few hours to get a
solution up & running...

However, the free Arduino IoT cloud is limited to 1 day of historical
data (I'd like months to a year). It also hides the bulk of the cloud
API (which is convenient) though I'm not sure yet how to handle
recovering from WiFi outages etc.

Internet searches uncover a ton of options many of which are far to
up-market for my purpose.

The free version of https://io.adafruit.com/ has 30 days of data
storage. And it has options for an easy to build simple data display.

Microsoft Azure has a free hub version: https://azure.microsoft.com/en-us/pricing/details/iot-hub/
I think this was called an S0 hub, but I don't see that name on the
page. It is far more limited than the payed options and does not provide
device provisioning.
Not sure about the storage however. You may have to pay for a storage
service, but with limited data, hot storage starts at $0.018/GB/month.
You do have to provide a credit card number when signing up to Azure, so
they can charge you as soon as you enable a paid service.
Not sure how to get data out of the Azure cloud (for free), my limited
Azure experience was mainly on the device side, sending data into the
cloud.

Amazon AWS only seems to provide time limited (12 months) free options.

Google has some free options, but it is not clear to me you can do
anything useful with only free stuff.


--
Stef

Almost nothing in Perl serves a single purpose.
-- Larry Wall in <199712040054.QAA13811@wall.org>

--- SoupGate-Win32 v1.05
* Origin: fsxNet Usenet Gateway (21:1/5)

Chris <chris.hoffmann@gmail.com> wrote:

It also appears there's another route using a local server (eg RPi) that receives the data (via NodeRed or similar) but then to access outside the local WiFi network means exposing a web server or similar - sounds like an extra tier or development and complication/maintenance.

To add to the options mentioned, you can also get a small VPS for maybe
$10-15 a year. You would of course need to configure it yourself, and
there's no resiliency in case of outages.

So not what I'd do for a commercial product (although I have seen fancy products with effectively this level of backend - and steered clear of them) but could be an option if this is a DIY project.

It prevents all the local network hassles, but does mean you have a box on
the internet that needs updating.

Another option is a hosted Docker container - Google Cloud Run, Azure and
IBM have free offerings, according to: https://www.reddit.com/r/docker/comments/c3urip/what_is_a_cheap_or_even_free_way_of_hosting_a/

Theo

--- SoupGate-Win32 v1.05
* Origin: fsxNet Usenet Gateway (21:1/5)

On 02/03/22 02:38, Chris wrote:

I'm setting up a household water-tank level sensor that I'd like to be able to monitor from (practically) anywhere to a mobile device or remote PCusing the above-mentioned ESP8266-based setup. The device is sampling at 5min intervals, each sample is a

single integer range 0 - 230 (cm of water height). Also toggling a 'heartbeat' boolean variable to indicate it's still alive.

Just getting stated with the ESP8266 and Arduino IoT Cloud via WiFi which was easy& convenient as it only took a few hours to get a solution up& running...

However, the free Arduino IoT cloud is limited to 1 day of historical data (I'd like months to a year). It also hides the bulk of the cloud API (which is convenient) though I'm not sure yet how to handle recovering from WiFi outages etc.

Internet searches uncover a ton of options many of which are far to up-market for my purpose.

It also appears there's another route using a local server (eg RPi) that receives the data (via NodeRed or similar) but then to access outside the local WiFi network means exposing a web server or similar - sounds like an extra tier or development and

complication/maintenance.

Any advice would be helpful...

--
Cheers,
Chris.

How about one of the 3G or 4G mobile data modules. Just call into it to
get the results, which can come from an rpi or similar board with a
serial or network interface. Only downside is the need for a sim card
but not expensive for low usage or bandwidth.

Or, a 3 or 4G router with network at one side could be made to work as
well, where you could just access an internal host directly via the
mobile network. Easy to secure as well...

Chris

--- SoupGate-Win32 v1.05
* Origin: fsxNet Usenet Gateway (21:1/5)

On Friday, 4 February 2022 at 10:57:22 UTC+11, chris wrote:

On 02/03/22 02:38, Chris wrote:

I'm setting up a household water-tank level sensor that I'd like to be able to monitor from (practically) anywhere to a mobile device or remote PCusing the above-mentioned ESP8266-based setup. The device is sampling at 5min intervals, each sample is

a single integer range 0 - 230 (cm of water height). Also toggling a 'heartbeat' boolean variable to indicate it's still alive.

Just getting stated with the ESP8266 and Arduino IoT Cloud via WiFi which was easy& convenient as it only took a few hours to get a solution up& running...

However, the free Arduino IoT cloud is limited to 1 day of historical data (I'd like months to a year). It also hides the bulk of the cloud API (which is convenient) though I'm not sure yet how to handle recovering from WiFi outages etc.

Internet searches uncover a ton of options many of which are far to up-market for my purpose.

It also appears there's another route using a local server (eg RPi) that receives the data (via NodeRed or similar) but then to access outside the local WiFi network means exposing a web server or similar - sounds like an extra tier or development

and complication/maintenance.

Any advice would be helpful...

--
Cheers,
Chris.

How about one of the 3G or 4G mobile data modules. Just call into it to
get the results, which can come from an rpi or similar board with a
serial or network interface. Only downside is the need for a sim card
but not expensive for low usage or bandwidth.

Or, a 3 or 4G router with network at one side could be made to work as
well, where you could just access an internal host directly via the
mobile network. Easy to secure as well...

Chris

Thanks to all for the tips.

This is not a commercial product, just an initial foray into this space and hopefully a more convenient way of keeping tabs on my water level without having to resort to uncovering the manhole & using a dipstick. We recently build a deck over the tank as
an outdoor area and even though I have a (screwed down) hatch in the deck over the manhole, it's an extra level of annoyance to get to it.

I'll look into the Adafruit offering and possibly the Azure one depending on what it offers.

Thanks for taking the time to reply.

--
Cheers,
Chris.

--- SoupGate-Win32 v1.05
* Origin: fsxNet Usenet Gateway (21:1/5)

Il 04/02/2022 00:57, chris ha scritto:

On 02/03/22 02:38, Chris wrote:

I'm setting up a household water-tank level sensor that I'd like to be
able to monitor from (practically) anywhere to a mobile device or
remote PCusing the above-mentioned ESP8266-based setup. The device is
sampling at 5min intervals, each sample is a single integer range 0 -
230 (cm of water height). Also toggling a 'heartbeat' boolean variable
to indicate it's still alive.

Just getting stated with the ESP8266 and Arduino IoT Cloud via WiFi
which was easy&  convenient as it only took a few hours to get a
solution up&  running...

However, the free Arduino IoT cloud is limited to 1 day of historical
data (I'd like months to a year). It also hides the bulk of the cloud
API (which is convenient) though I'm not sure yet how to handle
recovering from WiFi outages etc.

Internet searches uncover a ton of options many of which are far to
up-market for my purpose.

It also appears there's another route using a local server (eg RPi)
that receives the data (via NodeRed or similar) but then to access
outside the local WiFi network means exposing a web server or similar
- sounds like an extra tier or development and complication/maintenance.

Any advice would be helpful...

--
Cheers,
Chris.

How about one of the 3G or 4G mobile data modules. Just call into it to
get the results, which can come from an rpi or similar board with a
serial or network interface. Only downside is the need for a sim card
but not expensive for low usage or bandwidth.

What do you mean with "just call into it"?

Or, a 3 or 4G router with network at one side could be made to work as
well, where you could just access an internal host directly via the
mobile network. Easy to secure as well...

In my experience, it's not always possible to connect to a server that
is connected to a mobile 3G/4G network. Many mobile operators give
connectivity that is guaranteed to work for "standard web browsing".
Dynamic IP addresses, private IP addresses, firewall, NAT and similar
"filters" are enemis of servers.

This is the one of the reasons why IoT devices aren't server, but they
usually run a client (HTTP, MQTT or others similar) that connects to a
Cloud system.

--- SoupGate-Win32 v1.05
* Origin: fsxNet Usenet Gateway (21:1/5)

Il 04/02/2022 12:19, chris ha scritto:

On 02/04/22 07:34, pozz wrote:

Il 04/02/2022 00:57, chris ha scritto:

On 02/03/22 02:38, Chris wrote:

I'm setting up a household water-tank level sensor that I'd like to
be able to monitor from (practically) anywhere to a mobile device or
remote PCusing the above-mentioned ESP8266-based setup. The device is
sampling at 5min intervals, each sample is a single integer range 0 -
230 (cm of water height). Also toggling a 'heartbeat' boolean
variable to indicate it's still alive.

Just getting stated with the ESP8266 and Arduino IoT Cloud via WiFi
which was easy&  convenient as it only took a few hours to get a
solution up&  running...

However, the free Arduino IoT cloud is limited to 1 day of historical
data (I'd like months to a year). It also hides the bulk of the cloud
API (which is convenient) though I'm not sure yet how to handle
recovering from WiFi outages etc.

Internet searches uncover a ton of options many of which are far to
up-market for my purpose.

It also appears there's another route using a local server (eg RPi)
that receives the data (via NodeRed or similar) but then to access
outside the local WiFi network means exposing a web server or similar
- sounds like an extra tier or development and
complication/maintenance.

Any advice would be helpful...

--
Cheers,
Chris.

How about one of the 3G or 4G mobile data modules. Just call into it
to get the results, which can come from an rpi or similar board with a
serial or network interface. Only downside is the need for a sim card
but not expensive for low usage or bandwidth.

What do you mean with "just call into it"?

I bought a Netgear 4G router for evaluation and also to use as a backup
for the isp. That provides standard internet access just by fitting a
sim card. Really cheap for low bandwidth applications and it's always
on. Haven't tried it, but assume external access would be possible by
opening an incoming port to the local network. IP address is dynamic,
so you would need dynamic dns.org, whatever, but it should possible.
On the local subnet, a couple of pages of C on an rpi, to open a file
on the sd card and log tank values as needed. Login via ssh or run a
web server to access the stored data. Advantage is that it's a self
contained solution dependent only on a mobile account.

If it worked, I would agree with you. Howevere in this case you need a
server behind the router (what you say "opening an incoming port to the
local network"), you need NAT and so on. It could work, but it depends
on your mobile operator.

I know there are situations where this scenario is impossible, for
example if the mobile operator gives your device a *private* IP address.

Solutions like cloud etc really are overthinking what should be a simple problem to solve...

Sure.

Chris

Or, a 3 or 4G router with network at one side could be made to work as
well, where you could just access an internal host directly via the
mobile network. Easy to secure as well...

In my experience, it's not always possible to connect to a server that
is connected to a mobile 3G/4G network. Many mobile operators give
connectivity that is guaranteed to work for "standard web browsing".
Dynamic IP addresses, private IP addresses, firewall, NAT and similar
"filters" are enemis of servers.

This is the one of the reasons why IoT devices aren't server, but they
usually run a client (HTTP, MQTT or others similar) that connects to a
Cloud system.

--- SoupGate-Win32 v1.05
* Origin: fsxNet Usenet Gateway (21:1/5)

On 02/04/22 07:34, pozz wrote:

Il 04/02/2022 00:57, chris ha scritto:

On 02/03/22 02:38, Chris wrote:

I'm setting up a household water-tank level sensor that I'd like to
be able to monitor from (practically) anywhere to a mobile device or
remote PCusing the above-mentioned ESP8266-based setup. The device is
sampling at 5min intervals, each sample is a single integer range 0 -
230 (cm of water height). Also toggling a 'heartbeat' boolean
variable to indicate it's still alive.

Just getting stated with the ESP8266 and Arduino IoT Cloud via WiFi
which was easy& convenient as it only took a few hours to get a
solution up& running...

However, the free Arduino IoT cloud is limited to 1 day of historical
data (I'd like months to a year). It also hides the bulk of the cloud
API (which is convenient) though I'm not sure yet how to handle
recovering from WiFi outages etc.

Internet searches uncover a ton of options many of which are far to
up-market for my purpose.

It also appears there's another route using a local server (eg RPi)
that receives the data (via NodeRed or similar) but then to access
outside the local WiFi network means exposing a web server or similar
- sounds like an extra tier or development and complication/maintenance. >>>
Any advice would be helpful...

--
Cheers,
Chris.

How about one of the 3G or 4G mobile data modules. Just call into it
to get the results, which can come from an rpi or similar board with a
serial or network interface. Only downside is the need for a sim card
but not expensive for low usage or bandwidth.

What do you mean with "just call into it"?

I bought a Netgear 4G router for evaluation and also to use as a backup
for the isp. That provides standard internet access just by fitting a
sim card. Really cheap for low bandwidth applications and it's always
on. Haven't tried it, but assume external access would be possible by
opening an incoming port to the local network. IP address is dynamic,
so you would need dynamic dns.org, whatever, but it should possible.
On the local subnet, a couple of pages of C on an rpi, to open a file
on the sd card and log tank values as needed. Login via ssh or run a
web server to access the stored data. Advantage is that it's a self
contained solution dependent only on a mobile account.

Solutions like cloud etc really are overthinking what should be a simple problem to solve...

Chris

Or, a 3 or 4G router with network at one side could be made to work as
well, where you could just access an internal host directly via the
mobile network. Easy to secure as well...

In my experience, it's not always possible to connect to a server that
is connected to a mobile 3G/4G network. Many mobile operators give connectivity that is guaranteed to work for "standard web browsing".
Dynamic IP addresses, private IP addresses, firewall, NAT and similar "filters" are enemis of servers.

This is the one of the reasons why IoT devices aren't server, but they usually run a client (HTTP, MQTT or others similar) that connects to a
Cloud system.

--- SoupGate-Win32 v1.05
* Origin: fsxNet Usenet Gateway (21:1/5)

Chris <chris.hoffmann@gmail.com> wrote:

This is not a commercial product, just an initial foray into this space
and hopefully a more convenient way of keeping tabs on my water level
without having to resort to uncovering the manhole & using a dipstick. We recently build a deck over the tank as an outdoor area and even though I
have a (screwed down) hatch in the deck over the manhole, it's an extra
level of annoyance to get to it.

Depends how much this is a 'project' rather than an end goal, but Home Assistant will cover most of that.


HA on a Pi to do the web front end / phone app / etc

ESPHome on the ESP8266 to run the sensor node (just needs a config file if
the sensor type is already supported). HA auto-detects the ESP8266 and fetches/logs the sensor data.

Nabu Casa to provide the cloud connection so you can link back into your HA instance from outside your network. It also provides various cloud service interfacing that can't be done locally, eg Alexa and Google Assistant.


The first two are free, Nabu Casa is $6.50 per month. That's not super
cheap, but feeds back into funding Home Assistant. You are free to do your
own port-forwarding/dynamic DNS/etc setup if you'd rather not pay for that.

Theo

--- SoupGate-Win32 v1.05
* Origin: fsxNet Usenet Gateway (21:1/5)

On Saturday, 5 February 2022 at 03:45:13 UTC+11, Theo wrote:

Chris <chris.h...@gmail.com> wrote:

This is not a commercial product, just an initial foray into this space
and hopefully a more convenient way of keeping tabs on my water level without having to resort to uncovering the manhole & using a dipstick. We recently build a deck over the tank as an outdoor area and even though I have a (screwed down) hatch in the deck over the manhole, it's an extra level of annoyance to get to it.

Depends how much this is a 'project' rather than an end goal, but Home Assistant will cover most of that.

HA on a Pi to do the web front end / phone app / etc

ESPHome on the ESP8266 to run the sensor node (just needs a config file if the sensor type is already supported). HA auto-detects the ESP8266 and fetches/logs the sensor data.

Nabu Casa to provide the cloud connection so you can link back into your HA instance from outside your network. It also provides various cloud service interfacing that can't be done locally, eg Alexa and Google Assistant.

The first two are free, Nabu Casa is $6.50 per month. That's not super
cheap, but feeds back into funding Home Assistant. You are free to do your own port-forwarding/dynamic DNS/etc setup if you'd rather not pay for that.

Theo

Thanks! I'll have a look at this...

--
Cheers,
Chris.

--- SoupGate-Win32 v1.05
* Origin: fsxNet Usenet Gateway (21:1/5)

On Monday, 7 February 2022 at 09:35:55 UTC+11, Chris wrote:

On Saturday, 5 February 2022 at 03:45:13 UTC+11, Theo wrote:

Chris <> wrote:

This is not a commercial product, just an initial foray into this space and hopefully a more convenient way of keeping tabs on my water level without having to resort to uncovering the manhole & using a dipstick. We recently build a deck over the tank as an outdoor area and even though I have a (screwed down) hatch in the deck over the manhole, it's an extra level of annoyance to get to it.

Depends how much this is a 'project' rather than an end goal, but Home Assistant will cover most of that.

HA on a Pi to do the web front end / phone app / etc

ESPHome on the ESP8266 to run the sensor node (just needs a config file if the sensor type is already supported). HA auto-detects the ESP8266 and fetches/logs the sensor data.

Nabu Casa to provide the cloud connection so you can link back into your HA instance from outside your network. It also provides various cloud service interfacing that can't be done locally, eg Alexa and Google Assistant.

The first two are free, Nabu Casa is $6.50 per month. That's not super cheap, but feeds back into funding Home Assistant. You are free to do your own port-forwarding/dynamic DNS/etc setup if you'd rather not pay for that.

Theo

Thanks! I'll have a look at this...

--
Cheers,
Chris.

Thanks all for the advice. As previously indicated I have a demo working on the Arduino IoT cloud with it's attendant limitations.

I'm leaning toward a Mosquito MQTT client/broker solution as it seems a bit more lightweight than the ESPHome version (my old RPi 3B might cut the mustard but not sure).

I'm using a Adafruit Huzzah ESP8266 for the sensor and I have a couple of Omega2 embedded linux devices from <https://onion.io/resources/> that I can run the broker on.

Eventually it might even be possible to add in the Google Home mini to report the data (on the backburner for now).

--
Cheers,
Chris.

--- SoupGate-Win32 v1.05
* Origin: fsxNet Usenet Gateway (21:1/5)