But now interested in Pi.
So recommendations for advanced hardware appreciated.
Not just the Pi, but other peripherals and hardware (cases etc).
Will probably build many different kinds of projects.
I am an experienced programmer and electronics designer.
Dealt with many different Intel controller hardware items as well as
CP/M and Windows OS.
Programmed in PLM (Intel's language)for controllers.
But now interested in Pi.
So recommendations for advanced hardware appreciated.
Not just the Pi, but other peripherals and hardware (cases etc). Will probably build many different kinds of projects.
No baby steps please.
Advanced peripherals and hardware please.
Retirement is fun time ! (Or else !)
I am an experienced programmer and electronics designer.
Dealt with many different Intel controller hardware items as well as
CP/M and Windows OS.
Programmed in PLM (Intel's language)for controllers.
But now interested in Pi.
So recommendations for advanced hardware appreciated.
Not just the Pi, but other peripherals and hardware (cases etc).
Will probably build many different kinds of projects.
No baby steps please.
Advanced peripherals and hardware please.
Retirement is fun time ! (Or else !)
I have a yen to make a weather station as well, one day.
On Wed, 9 Sep 2020 12:01:57 -0700, Aioli <Aioli@Aioli.com> declaimed the following:
isBut now interested in Pi.
So recommendations for advanced hardware appreciated.
Not just the Pi, but other peripherals and hardware (cases etc).
Will probably build many different kinds of projects.
Problem is: any reply to your "advanced hardware" and "peripherals"
dependent upon WHAT you are building.set,
After all -- for someone wanting to replace a Windows desktop... An R-Pi 4B 8GB, an HDMI monitor, USB keyboard & mouse (I'd recommend saving a USB port by using something like a Logitech "Unifying" wireless keyboard
and mouse), and a USB(3) disk drive (on which one mounts /home, /tmp, /var and a swap file -- all the stuff that gets lots of changes that would wear out a uSD card) qualifies, with some box to put the drive and R-Pi into...
For someone trying to build a programmable Christmas tree lighting
a couple of long strings of NeoPixel LEDs, a big power supply (the LEDs
want power), almost any R-Pi with WiFi, and a case to hold the R-Pi and
power supply -- and an ability to code a simple web server application for controlling the LED sequencer program. Connect to R-Pi over WiFi to access LED configuration using a browser in a phone or tablet.
If you are doing projects that require measuring analog data (an oscilloscope perhaps) you will need a dedicated multi-port ADC chip (the
R-Pi does not have on-board ADC -- unlike the Beaglebone Black). You may
also want multi-line PWM chips as the R-Pi is a bit limited in that aspect too (software PWM is rate limited, and CPU heavy)
"The Natural Philosopher" <tnp@invalid.invalid> wrote in message news:rjcjf8$kee$1@dont-email.me...
I have a yen to make a weather station as well, one day.
I have a Davis Vantage Vue weather station (a remote sensor unit which measures wind speed/direction, rainfall amount/rate, outside temperature
and humidity) and communicates these to a base station by radio link (proprietary, not wifi). The base station has a USB output.
Some time, I will investigate uploading the data to an SQL database on
the server so a PHP program can extract and display graphs of specified parameters over a specified interval of time.
But now interested in Pi.
So recommendations for advanced hardware appreciated.
On 10/09/2020 10:55, NY wrote:
"The Natural Philosopher" <tnp@invalid.invalid> wrote in message
news:rjcjf8$kee$1@dont-email.me...
I have a yen to make a weather station as well, one day.
I have a Davis Vantage Vue weather station (a remote sensor unit which
measures wind speed/direction, rainfall amount/rate, outside temperature
and humidity) and communicates these to a base station by radio link
(proprietary, not wifi). The base station has a USB output.
I've got an Oregon Scientific weather station which can communicate with
an Android app via Bluetooth LE. Someone reverse engineered the protocol,
so I can use a Python program on a Pi to download temperature, humidity
and pressure readings. I combine these with readings from Htu21d and
BME280 I2C sensors on other Pi's which are around the house and shed, and
log to an SQLite database.
Some time, I will investigate uploading the data to an SQL database on
the server so a PHP program can extract and display graphs of specified
parameters over a specified interval of time.
I've got another Pi running nginx web server with uwsgi, so it can use a Python program (rather than PHP) to retrieve data from the SQL database
and generate HTML+javascript to plot it using Google charts.
... and if you want lots of digital and analogue i/o without adding
too many bits and pieces, get a BeagleBone Black! :-)
(I have several Pis and BBBs, horses for courses)
Yes the only bit I haven't cracked is running a process on the Pi that uploads data to the web server (maybe in daily batches, maybe every 10 minutes when a new entry is added to the local log file on the Pi) so as
to add it to the SQL table.
Yes I'd already identified that cron would be a good way of triggering
the upload. It's a matter of working out what can be run (via cron) at
the Pi which makes a remote web server add data to an SQL database held
on its server, preferably searching for each row of data that is about
to be added to check that it doesn't already have it - to make the
process resilient to temporary outages which would otherwise cause it to
miss data if there was no catchup mechanism.
I'll probably write the query-and-display software first, manually
uploading each month's data into the SQL table, and leave the automatic cron-driven uploading with resilience until later.
On Thu, 10 Sep 2020 17:22:55 +0100, NY wrote:
Yes the only bit I haven't cracked is running a process on the Pi thatThat sounds like a case for using a cron job. See "man cron" for how to manage crond, the cron daemon, and "man 5 crontab" for how to write a cron job script.
uploads data to the web server (maybe in daily batches, maybe every 10
minutes when a new entry is added to the local log file on the Pi) so as
to add it to the SQL table.
crond is a daemon process (i.e. started on boot and waits for stuff to
do) that looks for shell scripts to execute, runs any that it finds, and e-mails results and/or errors to the user who submitted the job.
Cron jobs can be run once every hour, day, week or month by putting a
script in the appropriate directory, e.g scripts in /etc/cron.daily are
run once a day, typically somewhat after midnight. In these cases the
script contains details of which user it should be run under and where to send any data written to stdout or stderr, with the default being to run
it under root and to send output to root.
Yes I'd already identified that cron would be a good way of triggering the >upload. It's a matter of working out what can be run (via cron) at the Pi >which makes a remote web server add data to an SQL database held on its >server, preferably searching for each row of data that is about to be added >to check that it doesn't already have it - to make the process resilient to >temporary outages which would otherwise cause it to miss data if there was
no catchup mechanism.
Yes the only bit I haven't cracked is running a process on the Pi that uploads data to the web server (maybe in daily batches, maybe every 10 minutes when a new entry is added to the local log file on the Pi) so as to add it to the SQL table. Doing it manually is easy enough from cPanel accessed by web page (LOAD DATA INFILE from a file on the Pi or Windows
PC)
Yes I'd already identified that cron would be a good way of triggering
the upload. It's a matter of working out what can be run (via cron) at
the Pi which makes a remote web server add data to an SQL database held
on its server, preferably searching for each row of data that is about
Yes the only bit I haven't cracked is running a process on the Pi that uploads data to the web server (maybe in daily batches, maybe every 10 minutes when a new entry is added to the local log file on the Pi) so as to add it to the SQL table. Doing it manually is easy enough from cPanel accessed by web page (LOAD DATA INFILE from a file on the Pi or Windows
PC) - I've done it for a big database of WWI soldiers' details that my parents gradually add to as they do research, so I save an Excel
spreadsheet
as a CSV file, wipe the SQL table of its existing data, and then upload the whole lot again (including additions and changes). It would be nice to find
a way of automating this, preferably so it only uploaded data that it
doesn't already have.
Yes I'd already identified that cron would be a good way of triggeringAs I said, 'curl' will send data to the server, as long as you can get
the upload. It's a matter of working out what can be run (via cron) at
the Pi which makes a remote web server add data to an SQL database held
on its server, preferably searching for each row of data that is about
to be added to check that it doesn't already have it - to make the
process resilient to temporary outages which would otherwise cause it to
miss data if there was no catchup mechanism.
It's on the "round tuit" list - a nice little refinement to what I can
do locally at present by searching the local CSV files that Cumulus
creates.
I'll probably write the query-and-display software first, manually
uploading each month's data into the SQL table, and leave the automatic cron-driven uploading with resilience until later.
Op 10-9-2020 21:38, NY wrote:
Yes I'd already identified that cron would be a good way of triggering
the upload. It's a matter of working out what can be run (via cron) at
the Pi which makes a remote web server add data to an SQL database
held on its server, preferably searching for each row of data that is
about
The normal solution for simple webhosting with no outside facing
database, is: make a PHP script on the web server that receives your
data, maybe via a form upload, maybe directly as a query parameter; that processes the data and connects to the database to save it. AKA build
your own API. Make sure you add some sort of login and/or secret key
and/or checksum. Also make sure that you sanitize your input before
throwing it to the db, even if it is just you!
On 10/09/2020 17:22, NY wrote:
Yes the only bit I haven't cracked is running a process on the Pi that
uploads data to the web server (maybe in daily batches, maybe every 10
minutes when a new entry is added to the local log file on the Pi) so
as to
add it to the SQL table. Doing it manually is easy enough from cPanel
accessed by web page (LOAD DATA INFILE from a file on the Pi or Windows
PC)
Ah so the database is on the web server. You will have find out what the database is (e.g. MySql), the IP address, and the username and password
to log on to it. You can then use a suitable Python package such as myslq-connector-python which can use that information to get a cursor on
to the db. Its then a case of issuing sql commands, in the same way as
for a local SQL db.
---druck
$query = "insert into data set";
$flag=0;
foreach($fields as $name) //read variables and add to query
{
if($flag) $query .=',';
if(isset($_GET[$name]))
$query.= sprintf(" %s='%s'",$name,$_POST[$name]);
else
$query.= sprintf(" %s='%s'",$name,"");
$flag++;
}
The Natural Philosopher wrote:funny, but obviously you don't understand sql as the sample code
$query = "insert into data set";
$flag=0;
foreach($fields as $name) //read variables and add to query
{
if($flag) $query .=',';
if(isset($_GET[$name]))
$query.= sprintf(" %s='%s'",$name,$_POST[$name]); >> else
$query.= sprintf(" %s='%s'",$name,"");
$flag++;
}
<https://xkcd.com/327>
On 11/09/2020 19:50, Andy Burns wrote:
The Natural Philosopher wrote:funny, but obviously you don't understand sql as the sample code
$query = "insert into data set";
$flag=0;
foreach($fields as $name) //read variables and add to query
{
if($flag) $query .=',';
if(isset($_GET[$name]))
$query.= sprintf(" %s='%s'",$name,$_POST[$name]); >>> else
$query.= sprintf(" %s='%s'",$name,"");
$flag++;
}
<https://xkcd.com/327>
specifically cannot do that kind of thing.
That is the reason why the SQL command is not passed.
And it is the reason why all the arguments are 'quoted'.
The Natural Philosopher <tnp@invalid.invalid> wrote:%s='%s'",$name,$_POST[$name]);
On 11/09/2020 19:50, Andy Burns wrote:
The Natural Philosopher wrote:
$query = "insert into data set";
$flag=0;
foreach($fields as $name) //read variables and add to query
{
if($flag) $query .=',';
if(isset($_GET[$name]))
$query.= sprintf("
funny, but obviously you don't understand sql as the sample code specifically cannot do that kind of thing.else
$query.= sprintf(" %s='%s'",$name,"");
$flag++;
}
<https://xkcd.com/327>
That is the reason why the SQL command is not passed.
And it is the reason why all the arguments are 'quoted'.
You mixed up _GET and _POST, there are no sanity checks and you just dump
it in the sql string. What if _POST[$name] starts with '; ?
The key is to
use mysqli_real_escape_string($dblink, $strval) or the equivalent for your db.
Doesn't mysql provide prepared statements with placeholders like
sqlite does ? Those are the safest and easiest way to put user date into
SQL.
On 12 Sep 2020 05:03:48 GMT
A. Dumas <alexandre@dumas.fr.invalid> wrote:
The Natural Philosopher <tnp@invalid.invalid> wrote:
On 11/09/2020 19:50, Andy Burns wrote:
The Natural Philosopher wrote:funny, but obviously you don't understand sql as the sample code
$query = "insert into data set";
$flag=0;
foreach($fields as $name) //read variables and add to query >>>>> {
if($flag) $query .=',';
if(isset($_GET[$name]))
$query.= sprintf(" %s='%s'",$name,$_POST[$name]);
else
$query.= sprintf(" %s='%s'",$name,"");
$flag++;
}
<https://xkcd.com/327>
specifically cannot do that kind of thing.
That is the reason why the SQL command is not passed.
And it is the reason why all the arguments are 'quoted'.
That is insufficient protection.
You mixed up _GET and _POST, there are no sanity checks and you just dump
it in the sql string. What if _POST[$name] starts with '; ?
Precisely.
The key is to
use mysqli_real_escape_string($dblink, $strval) or the equivalent for your >> db.
Doesn't mysql provide prepared statements with placeholders like
sqlite does ? Those are the safest and easiest way to put user date into
SQL.
On 12-09-2020 09:04, Ahem A Rivet's Shot wrote:There are a dozen ways of protecting against data corruption and malware available to any coder who can code and understands te REAL risk (as
Doesn't mysql provide prepared statements with placeholders like
sqlite does ? Those are the safest and easiest way to put user date into
SQL.
I don't know, maybe now it does. What I posted was my way of doing just
that.
Andy Burns wrote:
The Natural Philosopher wrote:
$query = "insert into data set";
$flag=0;
foreach($fields as $name) //read variables and add to query
{
if($flag) $query .=',';
if(isset($_GET[$name]))
$query.= sprintf(" %s='%s'",$name,$_POST[$name]); >>> else
$query.= sprintf(" %s='%s'",$name,"");
$flag++;
}
<https://xkcd.com/327>
funny, but obviously you don't understand sql
as the sample code specifically cannot do that kind of thing.
That is the reason why the SQL command is not passed.
And it is the reason why all the arguments are 'quoted'.
armchair security experts who think the NSA is going to hack an amateur weather database
If you think those mitigate SQL injection attacks you are badly
mistaken. Your code could use a fixed query string referencing
@variables which are initialised with the values passed.
The Natural Philosopher wrote:no. Try it
Andy Burns wrote:
The Natural Philosopher wrote:
$query = "insert into data set";
$flag=0;
foreach($fields as $name) //read variables and add to query
{
if($flag) $query .=',';
if(isset($_GET[$name]))
$query.= sprintf(" %s='%s'",$name,$_POST[$name]);
else
$query.= sprintf(" %s='%s'",$name,"");
$flag++;
}
<https://xkcd.com/327>
funny, but obviously you don't understand sql
I don't claim to specialize in SQL, but I've done my share, however it's
your bugs being discussed not mine.
as the sample code specifically cannot do that kind of thing.
Can I smuggle a single-quote and a semicolon into the $_POST[] array,
so that you concatenate it onto your query string thinking it's merely a value? Yes I can.
That is the reason why the SQL command is not passed.
And it is the reason why all the arguments are 'quoted'.
If you think those mitigate SQL injection attacks you are badly
mistaken. Your code could use a fixed query string referencing
@variables which are initialised with the values passed.
The Natural Philosopher wrote:
armchair security experts who think the NSA is going to hack an
amateur weather database
Who needs three letter agencies? There are enough bots out there
throwing attacks at every open port they can find to see what happens...
Andy Burns wrote:
Can I smuggle a single-quote and a semicolon into the $_POST[]
array, so that you concatenate it onto your query string thinking
it's merely a value? Yes I can.
no. Try it
Doesn't mysql provide prepared statements with placeholders like
sqlite does ? Those are the safest and easiest way to put user date into
SQL.
Is that the same as a prepared statement, as used by JDBC or (IIRC) ODBC interface modules? Prepared statements are designed specifically to
protect your database against injection attacks
If your DBMS supports database procedures, using them is also a good way
to avoid injection attacks.
Den 2020-09-12 kl. 13:35, skrev Martin Gregorie:
Is that the same as a prepared statement, as used by JDBC or (IIRC)
ODBC interface modules? Prepared statements are designed specifically
to protect your database against injection attacks
I don't think so. Prepared statement has been around longer than the web-form. Prepared statements are used if you don't want the database to create a new execution path every time you execute a statement where
only the parameters are changed. For at least Oracle, it is a way to
keep the statement in the SGA cache. It is all about performance.
That it is safer for webforms may be good - but not the reason it exists
<https://www.postgresql.org/docs/12/sql-prepare.html>
"PREPARE creates a prepared statement. A prepared statement is a
server-side object that can be used to optimize performance."
When I started coding professionally I learned that PREPARE is the way
to go - no concatenating strings to a statement. This was in 1997.
The code base suggest that Sql.Prepare in our sql module had been around
for many years already then.
If your DBMS supports database procedures, using them is also a good
way to avoid injection attacks.
And then you have a hard time to switch database.
Keep business logic in code and traceability in triggers.
At least I find that to be a sound principle.
If your DBMS supports database procedures, using them is also a good way
to avoid injection attacks.
On Sat, 12 Sep 2020 11:35:32 -0000 (UTC)
Martin Gregorie <martin@mydomain.invalid> wrote:
If your DBMS supports database procedures, using them is also a good way
to avoid injection attacks.
Also a good way to ensure vendor lock-in.
Den 2020-09-12 kl. 19:54, skrev Martin Gregorie:
<lots on VAX/VMS snipped>
Interesting, I did not know the VAX thing.
I'm pretty sure we used RDB on VAX before I was employed,
but I'm not sure if they used prepare or not.
My main reason to reply was just to say that I think performace was the reason for prepare - not to prevent injection.
IIRC the early ODBC modules didn't support prepared statements, but I
might be wrong about that.
My main reason to reply was just to say that I
think performace was the reasson for prepare -
not to prevent injection.
Den 2020-09-14 kl. 19:37, skrev Martin Gregorie:DescribeAndBindColumns.html>
On Sat, 12 Sep 2020 17:54:16 +0000, Martin Gregorie wrote:What? I've used ODBC and Prepare since 2012, which was when I ported our
IIRC the early ODBC modules didn't support prepared statements, but ITurns out they still don't, while JDBC modules do.
might be wrong about that.
WCS system to MS sqlserver
Here's an unrelated example of it being used as well
<https://www.easysoft.com/developer/languages/c/examples/
On Sat, 12 Sep 2020 17:54:16 +0000, Martin Gregorie wrote:
Turns out they still don't, while JDBC modules do.
IIRC the early ODBC modules didn't support prepared statements, but I
might be wrong about that.
| Sysop: | Keyop |
|---|---|
| Location: | Huddersfield, West Yorkshire, UK |
| Users: | 295 |
| Nodes: | 16 (2 / 14) |
| Uptime: | 09:47:22 |
| Calls: | 6,644 |
| Calls today: | 4 |
| Files: | 12,190 |
| Messages: | 5,326,468 |