*******************************************
5,000.00 Prize Offered
*******************************************
In the USA the working stiffs pay taxes. Then the US government
uses all that tax money to buy stuff, like this:
https://fdc.nal.usda.gov/download-datasets.html
This is a link to the latest and greatest food nutrition
data that is intended as an official reference source for the
food industry and other parties. A lot of the nutritional
labels one sees at the local supermarket likely contain
data obtained from this source.
The problem is that the data is FUCKING CORRUPTED.
That's correct. The data, in the CSV files (I haven't checked
the JSON), contains corruption that prevents it from being
successfully loaded into a database like MariaDB.
To make it easier, you will need to examine only three
core files from the "Full Download of All Data Types"
CSV package:
food.csv
food_nutrient.csv
nutrient.csv
The first person to successfully describe, in detail, the
corruption and his methods will get the prize.
*******************************************
5,000.00 Prize Offered
*******************************************
In the USA the working stiffs pay taxes. Then the US government
uses all that tax money to buy stuff, like this:
https://fdc.nal.usda.gov/download-datasets.html
This is a link to the latest and greatest food nutrition
data that is intended as an official reference source for the
food industry and other parties. A lot of the nutritional
labels one sees at the local supermarket likely contain
data obtained from this source.
The problem is that the data is FUCKING CORRUPTED.
To make it easier, you will need to examine only three
core files from the "Full Download of All Data Types"
CSV package:
food.csv
food_nutrient.csv
nutrient.csv
On Dec 1, 2023 at 5:54:01 AM EST, "Farley Flud" <ff@linux.rocks> wrote:
*******************************************
5,000.00 Prize Offered
*******************************************
Sure there is.
In the USA the working stiffs pay taxes. Then the US government
uses all that tax money to buy stuff, like this:
https://fdc.nal.usda.gov/download-datasets.html
This is a link to the latest and greatest food nutrition
data that is intended as an official reference source for the
food industry and other parties. A lot of the nutritional
labels one sees at the local supermarket likely contain
data obtained from this source.
The problem is that the data is FUCKING CORRUPTED.
No, the data is NOT corrupted. You are a clueless twat.
To make it easier, you will need to examine only three
core files from the "Full Download of All Data Types"
CSV package:
food.csv
food_nutrient.csv
nutrient.csv
I imported food.csv into SQLite and MS Access with no problems at all. I then
downloaded and installed MariaDB. You have to change the import parameters in
MariaDB. The Control Characters section contains invalid defaults for this file. The correct settings are:
Fields Terminated by ,
Fields Enclosed by " and check Optionally
Fields Escaped by blank
Lines Terminated by \n,because there are only LF, not CRLF in the csv. The default here was \r\n
All 2,021,091 records imported. I did all of this on my "Windows PC", which is
in fact a VM on my MacBook Pro which is running Unix (MacOS).
Now, a real man would embarrassed for assuming the file was "corrupted". The actual problem was PEBKAC. Will you be man enough to admit you were wrong?
No $5,000 reward needed. Proving - yet again - that you know nothing about SQL
databases was more than enough reward.
On 12/1/2023 11:58 AM, Tyrone wrote:
On Dec 1, 2023 at 5:54:01 AM EST, "Farley Flud" <ff@linux.rocks> wrote:
*******************************************
5,000.00 Prize Offered
*******************************************
Sure there is.
In the USA the working stiffs pay taxes. Then the US government
uses all that tax money to buy stuff, like this:
https://fdc.nal.usda.gov/download-datasets.html
This is a link to the latest and greatest food nutrition
data that is intended as an official reference source for the
food industry and other parties. A lot of the nutritional
labels one sees at the local supermarket likely contain
data obtained from this source.
The problem is that the data is FUCKING CORRUPTED.
No, the data is NOT corrupted. You are a clueless twat.
To make it easier, you will need to examine only three
core files from the "Full Download of All Data Types"
CSV package:
food.csv
food_nutrient.csv
nutrient.csv
I imported food.csv into SQLite and MS Access with no problems at all. I then
downloaded and installed MariaDB. You have to change the import parameters in
MariaDB. The Control Characters section contains invalid defaults for this >> file. The correct settings are:
Fields Terminated by ,
Fields Enclosed by " and check Optionally
Fields Escaped by blank
Lines Terminated by \n,because there are only LF, not CRLF in the csv. The >> default here was \r\n
All 2,021,091 records imported. I did all of this on my "Windows PC", which is
in fact a VM on my MacBook Pro which is running Unix (MacOS).
Good work. I did similar 4 years ago. Then I built a small Access
front-end to it:
https://imgur.com/a/RlibG2r
Note: that simple front-end is impossible to build with the LibreOffice crapware.
Now, a real man would embarrassed for assuming the file was "corrupted". The >> actual problem was PEBKAC. Will you be man enough to admit you were wrong? >>
No $5,000 reward needed. Proving - yet again - that you know nothing about SQL
databases was more than enough reward.
I think Feeb misused 'corrupted'. He likely means data that's missing
or violates referential integrity.
On Fri, 01 Dec 2023 16:58:16 +0000, Tyrone wrote:
All 2,021,091 records imported. I did all of this on my "Windows PC", which is
in fact a VM on my MacBook Pro which is running Unix (MacOS).
Nice. Except there are only 2,021,090 records.
FAIL!!! BIG FUCKING FAIL!!!
On Fri, 01 Dec 2023 16:58:16 +0000, Tyrone wrote:
All 2,021,091 records imported. I did all of this on my "Windows PC", which is
in fact a VM on my MacBook Pro which is running Unix (MacOS).
Nice. Except there are only 2,021,090 records.
FAIL!!! BIG FUCKING FAIL!!!
Ha, ha, ha, ha, ha, ha, ha, ha, ha, ha, ha, ha!
All 2,021,091 records imported. I did all of this on my "Windows PC", which is
in fact a VM on my MacBook Pro which is running Unix (MacOS).
I thought the file was corrupted and was impossible to import?
On Fri, 01 Dec 2023 19:27:37 +0000, Farley Flud wrote:
On Fri, 01 Dec 2023 16:58:16 +0000, Tyrone wrote:
All 2,021,091 records imported. I did all of this on my "Windows PC", which is
in fact a VM on my MacBook Pro which is running Unix (MacOS).
Nice. Except there are only 2,021,090 records.
FAIL!!! BIG FUCKING FAIL!!!
Ha, ha, ha, ha, ha, ha, ha, ha, ha, ha, ha, ha!
Wow! That Mr. Tyrone Shoelaces is quite the competent dude.
He manages to load 1,021,091 records from a CSV database that
only contains 1,021,090 records.
Furthermore, there are other VERY SUBTLE corruptions that
should have caused any self-respecting database import to
choke.
On Fri, 01 Dec 2023 19:35:14 +0000, Tyrone wrote:
I thought the file was corrupted and was impossible to import?
It is.
But you managed to import it and with an extra record to boot.
Where'd that extra record come from?
Mr. Tyrone Shoelaces pulled it out of his fucking ass.
That's where it came from.
Now, instead of attempting to conceal your stupidity, locate and
describe the corruption.
On Fri, 01 Dec 2023 16:58:16 +0000, Tyrone wrote:
All 2,021,091 records imported. I did all of this on my "Windows PC", which is
in fact a VM on my MacBook Pro which is running Unix (MacOS).
Nice. Except there are only 2,021,090 records.
FAIL!!! BIG FUCKING FAIL!!!
Ha, ha, ha, ha, ha, ha, ha, ha, ha, ha, ha, ha!
On Fri, 01 Dec 2023 16:58:16 +0000, Tyrone wrote:
All 2,021,091 records imported. I did all of this on my "Windows PC", which is
in fact a VM on my MacBook Pro which is running Unix (MacOS).
Nice. Except there are only 2,021,090 records.
FAIL!!! BIG FUCKING FAIL!!!
Ha, ha, ha, ha, ha, ha, ha, ha, ha, ha, ha, ha!
On 12/1/2023 2:27 PM, Farley Flud wrote:
On Fri, 01 Dec 2023 16:58:16 +0000, Tyrone wrote:
All 2,021,091 records imported. I did all of this on my "Windows PC", which is
in fact a VM on my MacBook Pro which is running Unix (MacOS).
Nice. Except there are only 2,021,090 records.
FAIL!!! BIG FUCKING FAIL!!!
Ha, ha, ha, ha, ha, ha, ha, ha, ha, ha, ha, ha!
Wrong, dunce. In food.csv there are 2021092 records: a header row +
2021091 rows of data.
Line 41944 is just: ","","2019-04-01"
because line 41943 has an incorrect LF after the description.
Bad data, but line 41944 is still a record.
Feeb fail.
Also, lines 1171659-1171661 each contain a few undefined characters in
the Description field, but the data imports cleanly and isn't truncated.
https://imgur.com/a/JO6iEhy
Feeb fail.
Line 41944 is just: ","","2019-04-01"
because line 41943 has an incorrect LF after the description.
Bad data, but line 41944 is still a record.
Also, lines 1171659-1171661 each contain a few undefined characters in
the Description field, but the data imports cleanly and isn't truncated.
Microslop's native encoding for text is UTF-16,
You FAILED! You're FIRED! Hit the road, losers!
DFS' "undefined characters" are UTF-8 surrogate pairs, not UTF-16.
"Fatty Acid Content of Retail Cow’s Milk...
When I edit ( Visual Studio 2019 ) "Food.CSV"
( a 165 MegaByte text file in "FoodData_Central_csv_2023-10-26" )
& "Find ALL" "Fatty Acid Content" in "*.CSV" files, I see:
"Fatty Acid Content of Retail Cow’s Milk [. . .]",
That is fucking UTF-16 encoding!
DFS' "undefined characters" are UTF-8 surrogate pairs, not UTF-16.
"Fatty Acid Content of Retail Cow’s Milk in the Northeastern United States—What’s in It for the Consumer?",
I imported food.csv into SQLite and MS Access with no problems at all.
Sysop: | Keyop |
---|---|
Location: | Huddersfield, West Yorkshire, UK |
Users: | 300 |
Nodes: | 16 (2 / 14) |
Uptime: | 06:46:26 |
Calls: | 6,706 |
Files: | 12,236 |
Messages: | 5,350,566 |