• Data Challenge for GNU/Linux Gurus (Prize Offered)

    From Farley Flud@21:1/5 to All on Fri Dec 1 10:54:01 2023
    *******************************************
    5,000.00 Prize Offered
    *******************************************

    In the USA the working stiffs pay taxes. Then the US government
    uses all that tax money to buy stuff, like this:

    https://fdc.nal.usda.gov/download-datasets.html

    This is a link to the latest and greatest food nutrition
    data that is intended as an official reference source for the
    food industry and other parties. A lot of the nutritional
    labels one sees at the local supermarket likely contain
    data obtained from this source.

    The problem is that the data is FUCKING CORRUPTED.

    That's correct. The data, in the CSV files (I haven't checked
    the JSON), contains corruption that prevents it from being
    successfully loaded into a database like MariaDB.

    For one thing, the character set is not UTF-8. The data
    uses the outmoded Latin1, or ISO-8859-1, character set!

    But that's not too important. The corruption is the show
    stopper.

    You job is to locate and describe this corruption using
    the wonderful and competent GNU/Linux tools.

    Then, send the USDA a very nasty email about how they are
    wasting the taxpayer's bucks.

    To make it easier, you will need to examine only three
    core files from the "Full Download of All Data Types"
    CSV package:

    food.csv
    food_nutrient.csv
    nutrient.csv

    The first person to successfully describe, in detail, the
    corruption and his methods will get the prize.

    Find the corruption. With GNU/Linux it will be a snap.

    How about with Microslop Winblows?

    Ha, ha, ha, ha, ha, ha, ha, ha, ha, ha, ha!

    An OS made for retards and idiots will not cut it.

    Just ask Bing.

    Ha, ha, ha, ha, ha, ha, ha, ha, ha, ha, ha!

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From DFS@21:1/5 to Farley Flud on Fri Dec 1 09:16:13 2023
    On 12/1/2023 5:54 AM, Farley Flud wrote:
    *******************************************
    5,000.00 Prize Offered
    *******************************************

    In the USA the working stiffs pay taxes. Then the US government
    uses all that tax money to buy stuff, like this:

    https://fdc.nal.usda.gov/download-datasets.html

    This is a link to the latest and greatest food nutrition
    data that is intended as an official reference source for the
    food industry and other parties. A lot of the nutritional
    labels one sees at the local supermarket likely contain
    data obtained from this source.

    The problem is that the data is FUCKING CORRUPTED.

    That's correct. The data, in the CSV files (I haven't checked
    the JSON), contains corruption that prevents it from being
    successfully loaded into a database like MariaDB.


    To make it easier, you will need to examine only three
    core files from the "Full Download of All Data Types"
    CSV package:

    food.csv
    food_nutrient.csv
    nutrient.csv

    The first person to successfully describe, in detail, the
    corruption and his methods will get the prize.


    Put the money in an escrow account with an acceptable arbiter and I'll
    find that bad data, plus more I'm sure.

    But remember, you already owe big bucks to -hh for a previous online
    challenge.

    You'll never pay a dime, of course. You're broke, you're a liar and
    you're a welcher... in addition to all your other shit qualities.

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From Tyrone@21:1/5 to Farley Flud on Fri Dec 1 16:58:16 2023
    On Dec 1, 2023 at 5:54:01 AM EST, "Farley Flud" <ff@linux.rocks> wrote:

    *******************************************
    5,000.00 Prize Offered
    *******************************************

    Sure there is.

    In the USA the working stiffs pay taxes. Then the US government
    uses all that tax money to buy stuff, like this:

    https://fdc.nal.usda.gov/download-datasets.html

    This is a link to the latest and greatest food nutrition
    data that is intended as an official reference source for the
    food industry and other parties. A lot of the nutritional
    labels one sees at the local supermarket likely contain
    data obtained from this source.

    The problem is that the data is FUCKING CORRUPTED.

    No, the data is NOT corrupted. You are a clueless twat.

    To make it easier, you will need to examine only three
    core files from the "Full Download of All Data Types"
    CSV package:

    food.csv
    food_nutrient.csv
    nutrient.csv

    I imported food.csv into SQLite and MS Access with no problems at all. I then downloaded and installed MariaDB. You have to change the import parameters in MariaDB. The Control Characters section contains invalid defaults for this file. The correct settings are:

    Fields Terminated by ,
    Fields Enclosed by " and check Optionally
    Fields Escaped by blank
    Lines Terminated by \n,because there are only LF, not CRLF in the csv. The default here was \r\n

    All 2,021,091 records imported. I did all of this on my "Windows PC", which is in fact a VM on my MacBook Pro which is running Unix (MacOS).

    Now, a real man would embarrassed for assuming the file was "corrupted". The actual problem was PEBKAC. Will you be man enough to admit you were wrong?

    No $5,000 reward needed. Proving - yet again - that you know nothing about SQL databases was more than enough reward.

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From DFS@21:1/5 to Tyrone on Fri Dec 1 12:47:49 2023
    On 12/1/2023 11:58 AM, Tyrone wrote:
    On Dec 1, 2023 at 5:54:01 AM EST, "Farley Flud" <ff@linux.rocks> wrote:

    *******************************************
    5,000.00 Prize Offered
    *******************************************

    Sure there is.

    In the USA the working stiffs pay taxes. Then the US government
    uses all that tax money to buy stuff, like this:

    https://fdc.nal.usda.gov/download-datasets.html

    This is a link to the latest and greatest food nutrition
    data that is intended as an official reference source for the
    food industry and other parties. A lot of the nutritional
    labels one sees at the local supermarket likely contain
    data obtained from this source.

    The problem is that the data is FUCKING CORRUPTED.

    No, the data is NOT corrupted. You are a clueless twat.

    To make it easier, you will need to examine only three
    core files from the "Full Download of All Data Types"
    CSV package:

    food.csv
    food_nutrient.csv
    nutrient.csv

    I imported food.csv into SQLite and MS Access with no problems at all. I then
    downloaded and installed MariaDB. You have to change the import parameters in
    MariaDB. The Control Characters section contains invalid defaults for this file. The correct settings are:

    Fields Terminated by ,
    Fields Enclosed by " and check Optionally
    Fields Escaped by blank
    Lines Terminated by \n,because there are only LF, not CRLF in the csv. The default here was \r\n

    All 2,021,091 records imported. I did all of this on my "Windows PC", which is
    in fact a VM on my MacBook Pro which is running Unix (MacOS).

    Good work. I did similar 4 years ago. Then I built a small Access
    front-end to it:

    https://imgur.com/a/RlibG2r

    Note: that simple front-end is impossible to build with the LibreOffice crapware.



    Now, a real man would embarrassed for assuming the file was "corrupted". The actual problem was PEBKAC. Will you be man enough to admit you were wrong?

    No $5,000 reward needed. Proving - yet again - that you know nothing about SQL
    databases was more than enough reward.


    I think Feeb misused 'corrupted'. He likely means data that's missing
    or violates referential integrity.

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From Tyrone@21:1/5 to DFS on Fri Dec 1 18:59:45 2023
    On Dec 1, 2023 at 12:47:49 PM EST, "DFS" <nospam@dfs.com> wrote:

    On 12/1/2023 11:58 AM, Tyrone wrote:
    On Dec 1, 2023 at 5:54:01 AM EST, "Farley Flud" <ff@linux.rocks> wrote:

    *******************************************
    5,000.00 Prize Offered
    *******************************************

    Sure there is.

    In the USA the working stiffs pay taxes. Then the US government
    uses all that tax money to buy stuff, like this:

    https://fdc.nal.usda.gov/download-datasets.html

    This is a link to the latest and greatest food nutrition
    data that is intended as an official reference source for the
    food industry and other parties. A lot of the nutritional
    labels one sees at the local supermarket likely contain
    data obtained from this source.

    The problem is that the data is FUCKING CORRUPTED.

    No, the data is NOT corrupted. You are a clueless twat.

    To make it easier, you will need to examine only three
    core files from the "Full Download of All Data Types"
    CSV package:

    food.csv
    food_nutrient.csv
    nutrient.csv

    I imported food.csv into SQLite and MS Access with no problems at all. I then
    downloaded and installed MariaDB. You have to change the import parameters in
    MariaDB. The Control Characters section contains invalid defaults for this >> file. The correct settings are:

    Fields Terminated by ,
    Fields Enclosed by " and check Optionally
    Fields Escaped by blank
    Lines Terminated by \n,because there are only LF, not CRLF in the csv. The >> default here was \r\n

    All 2,021,091 records imported. I did all of this on my "Windows PC", which is
    in fact a VM on my MacBook Pro which is running Unix (MacOS).

    Good work. I did similar 4 years ago. Then I built a small Access
    front-end to it:

    https://imgur.com/a/RlibG2r

    Note: that simple front-end is impossible to build with the LibreOffice crapware.



    Now, a real man would embarrassed for assuming the file was "corrupted". The >> actual problem was PEBKAC. Will you be man enough to admit you were wrong? >>
    No $5,000 reward needed. Proving - yet again - that you know nothing about SQL
    databases was more than enough reward.


    I think Feeb misused 'corrupted'. He likely means data that's missing
    or violates referential integrity.

    Maybe, but if he is confusing "corrupted .csv file" with referential integrity then he is even more confused than is generally believed.

    And he clearly said the files are corrupted:

    Begin quote--

    "That's correct. The data, in the CSV files (I haven't checked
    the JSON), contains corruption that prevents it from being
    successfully loaded into a database like MariaDB.

    For one thing, the character set is not UTF-8. The data
    uses the outmoded Latin1, or ISO-8859-1, character set!

    But that's not too important. The corruption is the show
    stopper.

    You job is to locate and describe this corruption using
    the wonderful and competent GNU/Linux tools."

    End quote--

    The character set has nothing to do with anything. The file contains readable ASCII text. There are no hidden/non-printable/control characters in the file.

    Referential integrity is not a feature of .csv files. These files may have
    been created in Excel and be intended for nothing more than Excel. A competent DBA will be able to get this stuff into the system with referential integrity.


    The fact is, the .csv files are not corrupted. They can be imported into a database (or 3 in my case). What you do with it afterwards is not the concern of the people who created the .csv files. I know this because I have created thousands of .csv files over the years to send to clients that need to import it into whatever. I do not ask and do not care what they are going to do with it.

    They ask for .csv files and I send them .csv files. Case closed.

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From Tyrone@21:1/5 to Farley Flud on Fri Dec 1 19:35:14 2023
    On Dec 1, 2023 at 2:27:37 PM EST, "Farley Flud" <ff@linux.rocks> wrote:

    On Fri, 01 Dec 2023 16:58:16 +0000, Tyrone wrote:


    All 2,021,091 records imported. I did all of this on my "Windows PC", which is
    in fact a VM on my MacBook Pro which is running Unix (MacOS).


    Nice. Except there are only 2,021,090 records.

    FAIL!!! BIG FUCKING FAIL!!!

    That's it? That's all you have to say? I thought the file was corrupted and was impossible to import?

    HUGE FUCKING FAIL ON YOUR PART!!!

    No admission that you were wrong? OF COURSE NOT.

    HUGE FUCKING FAIL NUMBER 2 ON YOUR PART!!!

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From Farley Flud@21:1/5 to Farley Flud on Fri Dec 1 19:57:36 2023
    On Fri, 01 Dec 2023 19:27:37 +0000, Farley Flud wrote:

    On Fri, 01 Dec 2023 16:58:16 +0000, Tyrone wrote:


    All 2,021,091 records imported. I did all of this on my "Windows PC", which is
    in fact a VM on my MacBook Pro which is running Unix (MacOS).


    Nice. Except there are only 2,021,090 records.

    FAIL!!! BIG FUCKING FAIL!!!

    Ha, ha, ha, ha, ha, ha, ha, ha, ha, ha, ha, ha!


    Wow! That Mr. Tyrone Shoelaces is quite the competent dude.
    He manages to load 1,021,091 records from a CSV database that
    only contains 1,021,090 records.

    Whew! Impressive!

    Ha, ha, ha, ha, ha, ha, ha, ha, ha, ha, ha, ha!

    FAIL!!! BIG FUCKING FAIL!!!

    Furthermore, there are other VERY SUBTLE corruptions that
    should have caused any self-respecting database import to
    choke.

    But you'll NEVER find those. NEVER! Because you are a stupid,
    incompetent dumb-fuck and now the whole world knows it.

    Ha, ha, ha, ha, ha, ha, ha, ha, ha, ha, ha, ha!

    Now, if you publicly admit that you are an idiot, I just
    MAY reveal those VERY SUBTLE other corruptions.

    Yes, maybe.

    But I'll give you another chance to find them.

    But you won't. EVER!

    Ha, ha, ha, ha, ha, ha, ha, ha, ha, ha, ha, ha!

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From Farley Flud@21:1/5 to Tyrone on Fri Dec 1 19:27:37 2023
    On Fri, 01 Dec 2023 16:58:16 +0000, Tyrone wrote:


    All 2,021,091 records imported. I did all of this on my "Windows PC", which is
    in fact a VM on my MacBook Pro which is running Unix (MacOS).


    Nice. Except there are only 2,021,090 records.

    FAIL!!! BIG FUCKING FAIL!!!

    Ha, ha, ha, ha, ha, ha, ha, ha, ha, ha, ha, ha!

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From Farley Flud@21:1/5 to Tyrone on Fri Dec 1 20:42:50 2023
    On Fri, 01 Dec 2023 19:35:14 +0000, Tyrone wrote:


    I thought the file was corrupted and was impossible to import?


    It is.

    But you managed to import it and with an extra record to boot.

    Where'd that extra record come from?

    Mr. Tyrone Shoelaces pulled it out of his fucking ass.

    That's where it came from.

    Now, instead of attempting to conceal your stupidity, locate and
    describe the corruption.

    Hint: there is one glaring corruption and about a dozen VERY
    SUBTLE corruptions. But any of them should have caused
    an import failure.

    I can easily predict that you will NEVER find them.

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From Tyrone@21:1/5 to Farley Flud on Fri Dec 1 20:18:58 2023
    On Dec 1, 2023 at 2:57:36 PM EST, "Farley Flud" <ff@linux.rocks> wrote:

    On Fri, 01 Dec 2023 19:27:37 +0000, Farley Flud wrote:

    On Fri, 01 Dec 2023 16:58:16 +0000, Tyrone wrote:


    All 2,021,091 records imported. I did all of this on my "Windows PC", which is
    in fact a VM on my MacBook Pro which is running Unix (MacOS).


    Nice. Except there are only 2,021,090 records.

    FAIL!!! BIG FUCKING FAIL!!!

    Ha, ha, ha, ha, ha, ha, ha, ha, ha, ha, ha, ha!


    Wow! That Mr. Tyrone Shoelaces is quite the competent dude.
    He manages to load 1,021,091 records from a CSV database that
    only contains 1,021,090 records.

    There are 1,021,091 records. The header record counts and is easily deleted with:

    DELETE FROM food
    WHERE FDC_ID = 'fdc_id'

    But you knew that, right? That is very basic SQL.

    Furthermore, there are other VERY SUBTLE corruptions that
    should have caused any self-respecting database import to
    choke.

    No, there are no corruptions. Is MariaDB not a self-respecting DB? That's what I used. That's what YOU said could not import the file. You are wrong. Again.


    There you have it folks. Farley Fucktard is proven wrong, yet he continues his song and dance act.

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From Tyrone@21:1/5 to Farley Flud on Fri Dec 1 21:10:44 2023
    On Dec 1, 2023 at 3:42:50 PM EST, "Farley Flud" <fflud@gnu.rocks> wrote:

    On Fri, 01 Dec 2023 19:35:14 +0000, Tyrone wrote:


    I thought the file was corrupted and was impossible to import?


    It is.

    But you managed to import it and with an extra record to boot.

    Where'd that extra record come from?

    Mr. Tyrone Shoelaces pulled it out of his fucking ass.

    That's where it came from.

    ITS THE HEADER RECORD, DIPSHIT. It IS a record in the .csv file.


    Now, instead of attempting to conceal your stupidity, locate and
    describe the corruption.

    There is NO corruption. The file imported. You are wrong. Did you even try the settings I told you to use?

    Got any more stupid questions you want to ask? You are 0 out of 1000 at this point. But hey, go for 0 out of 1001.

    You STILL can't admit that you have no clue about any of this. I imported the file - into the DB that you said could not import it - and PROVED you are wrong. Grow up and learn from it.

    Idiot.

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From DFS@21:1/5 to Farley Flud on Sat Dec 2 00:15:27 2023
    On 12/1/2023 2:27 PM, Farley Flud wrote:
    On Fri, 01 Dec 2023 16:58:16 +0000, Tyrone wrote:


    All 2,021,091 records imported. I did all of this on my "Windows PC", which is
    in fact a VM on my MacBook Pro which is running Unix (MacOS).


    Nice. Except there are only 2,021,090 records.

    FAIL!!! BIG FUCKING FAIL!!!

    Ha, ha, ha, ha, ha, ha, ha, ha, ha, ha, ha, ha!


    Wrong, dunce. In food.csv there are 2021092 records: a header row +
    2021091 rows of data.

    Line 41944 is just: ","","2019-04-01"

    because line 41943 has an incorrect LF after the description.

    Bad data, but line 41944 is still a record.

    Feeb fail.


    Also, lines 1171659-1171661 each contain a few undefined characters in
    the Description field, but the data imports cleanly and isn't truncated.

    https://imgur.com/a/JO6iEhy

    Feeb fail.

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From DFS@21:1/5 to Farley Flud on Sat Dec 2 00:17:31 2023
    On 12/1/2023 2:27 PM, Farley Flud wrote:
    On Fri, 01 Dec 2023 16:58:16 +0000, Tyrone wrote:


    All 2,021,091 records imported. I did all of this on my "Windows PC", which is
    in fact a VM on my MacBook Pro which is running Unix (MacOS).


    Nice. Except there are only 2,021,090 records.

    FAIL!!! BIG FUCKING FAIL!!!

    Ha, ha, ha, ha, ha, ha, ha, ha, ha, ha, ha, ha!



    Wrong, dunce. In food.csv there are 2021092 records: a header row +
    2021091 rows of data.

    Line 41943 has an incorrect LF after the description, so
    line 41944 is just: ","","2019-04-01"

    It's bad data, but it's a record.

    Feeb fail.


    Also, lines 1171659-1171661 each contain a few undefined characters in
    the Description field, but the data imports cleanly and isn't truncated.

    https://imgur.com/a/JO6iEhy

    Feeb fail.

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From Tyrone@21:1/5 to DFS on Sat Dec 2 05:45:51 2023
    On Dec 2, 2023 at 12:15:27 AM EST, "DFS" <nospam@dfs.com> wrote:

    On 12/1/2023 2:27 PM, Farley Flud wrote:
    On Fri, 01 Dec 2023 16:58:16 +0000, Tyrone wrote:


    All 2,021,091 records imported. I did all of this on my "Windows PC", which is
    in fact a VM on my MacBook Pro which is running Unix (MacOS).


    Nice. Except there are only 2,021,090 records.

    FAIL!!! BIG FUCKING FAIL!!!

    Ha, ha, ha, ha, ha, ha, ha, ha, ha, ha, ha, ha!


    Wrong, dunce. In food.csv there are 2021092 records: a header row +
    2021091 rows of data.

    Line 41944 is just: ","","2019-04-01"

    because line 41943 has an incorrect LF after the description.

    Bad data, but line 41944 is still a record.

    Feeb fail.


    Also, lines 1171659-1171661 each contain a few undefined characters in
    the Description field, but the data imports cleanly and isn't truncated.

    https://imgur.com/a/JO6iEhy

    Feeb fail.

    Of course he fails. All he can do is compile apps via canned script files.
    He has no idea how to actually USE the stuff he compiles.

    I was also thinking that he was assuming that the records with multiple embedded quotes would be a problem. Search the food.csv file for """. There are over 5,000 records with these.

    AGAIN, the file is not corrupted. Notice that Feeb has abandoned this thread and has moved on to how he magically compiles stuff.

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From Farley Flud@21:1/5 to DFS on Sat Dec 2 10:17:03 2023
    On Sat, 2 Dec 2023 00:15:27 -0500, DFS wrote:


    Line 41944 is just: ","","2019-04-01"

    because line 41943 has an incorrect LF after the description.

    Bad data, but line 41944 is still a record.


    No fucking goddamned way.

    A valid record contains all the fields that are described by
    the header. Do lines 41943 and 41944 contain fields that
    match the header fields?

    No they fucking goddamned do not, and any pro database, like
    MariaDB, will choke right there.

    You FAIL, and you FAIL BIG!


    Also, lines 1171659-1171661 each contain a few undefined characters in
    the Description field, but the data imports cleanly and isn't truncated.


    Also BIG FUCKING FAIL!

    The characters are not "undefined." They are "inconsistent."

    The overall character encoding is ASCII but 11 lines contain
    characters that are encoded as UTF-16.

    A database is *always* created by specifying the encoding:

    create database fdc2023 character set = 'ascii';

    When any pro database, like MariaDB, expects ASCII but
    encounters a UTF-16 character it will choke.

    Originally, I thought that the encoding was Latin1,
    or ISO-8859-1:

    create database fdc2023 character set = 'latin1';

    Now the data will import cleanly but the strings will
    be malformed. They will be malformed because Latin1
    does not equal UTF-16.

    Where are these lines?

    After fixing line 41943 by removing the misplaced
    line feed, one will obtain a fixed csv file with new
    line numbers. Using these new line numbers the
    following lines contain UTF-16 encoded chars:

    1171658
    1171659
    1171660
    1636113
    1722798
    1722807
    1875582
    2003086
    2003093
    2003094
    2003106

    I'll let you, and that other retard Tyrone, locate
    the actual UTF-16 encoded characters within those
    lines.

    How did UTF-16 get mixed in with ASCII data? I can
    only guess, but since Microslop's native encoding for
    text is UTF-16, somehow the conversion got messed up
    (as expected with Microslop junk).

    If they had been using GNU/Linux, with native UTF-8,
    this NEVER would have happened.

    But you and Tyrone are both FIRED. Get out! There's
    no room for incompetent retards in this business.

    You FAILED! You're FIRED! Hit the road, losers!

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From Relf@21:1/5 to All on Sat Dec 2 03:19:24 2023
    DFS' "undefined characters" are UTF-8 surrogate pairs, not UTF-16.

    When I edit ( Visual Studio 2019 ) "Food.CSV"
    ( a 165 MegaByte text file in "FoodData_Central_csv_2023-10-26" )
    & "Find ALL" "Fatty Acid Content" in "*.CSV" files, I see:

    "Fatty Acid Content of Retail Cow’s Milk in the Northeastern United States—What’s in It for the Consumer?",

    Unlike you guys, with your outdated/unruly databases,
    DevStudio ( now Visual Studio ) was Unicode aware in 1988.

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From Relf@21:1/5 to All on Sat Dec 2 03:36:00 2023
    Microslop's native encoding for text is UTF-16,

    UTF-8 is the default character set for all file names everywhere,
    including NTFS (Windows), Linux, Chrome, HTML & JavaScript.

    You FAILED! You're FIRED! Hit the road, losers!

    Sadly, you're par for the course.

    P.S. Using UTF-16 ( as Visual C++ does ), _One_ emoji ligature might contain _Four_ different surrogate pairs, each joined via
    a ZWJ ( Zero Width Joiner, U+200D ) code point.

    For example: the 👨‍👩‍👧‍👦 emoji is 24 bytes long: 👨 ZWJ 👩 ZWJ 👧 ZWJ 👦 .

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From Farley Flud@21:1/5 to ScumRelf on Sat Dec 2 11:56:44 2023
    On Sat, ScumRelf wrote:


    DFS' "undefined characters" are UTF-8 surrogate pairs, not UTF-16.

    "Fatty Acid Content of Retail Cow’s Milk...


    Fuck you! You worthless piece of moldy dog shit!

    That is fucking UTF-16 encoding!

    The UTF-16 encoding for the "Right Single Quotation Mark"
    is hexadecimal 8217.

    Take your "surrogate pairs" and stuff them up your
    clyster pipes, you fucking decrepit, retarded faggot.

    You dare to contradict me and I'll slam you into
    fucking oblivion. Fucking worthless worm scum.

    One punch with my fist into your ugly, bloated face
    will put you in your rightful place forever.

    It'll put an end to your egregious bullshit forever.

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From Relf@21:1/5 to All on Sat Dec 2 04:44:15 2023
    You ( Detroit's finest ) replied to me
    ( in Seattle, across the pond from Microsoft ):
    When I edit ( Visual Studio 2019 ) "Food.CSV"
    ( a 165 MegaByte text file in "FoodData_Central_csv_2023-10-26" )
    & "Find ALL" "Fatty Acid Content" in "*.CSV" files, I see:

    "Fatty Acid Content of Retail Cow’s Milk [. . .]",

    That is fucking UTF-16 encoding!

    Edit "Food.CSV" with a _BINARY_ editor.
    Does each letter consume 2 bytes ?! if not, it's not UTF-16.

    Unlike you guys, with your outdated/unruly databases,
    DevStudio ( now Visual Studio ) was Unicode aware in 1988.

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From Farley Flud@21:1/5 to Relf on Sat Dec 2 12:45:55 2023
    On Sat, 02 Dec 2023 03:19:24 -0800 (Seattle), Relf wrote:

    DFS' "undefined characters" are UTF-8 surrogate pairs, not UTF-16.

    "Fatty Acid Content of Retail Cow’s Milk in the Northeastern United States—What’s in It for the Consumer?",

    ===================================^

    Fuck you! You worthless piece of moldy dog shit!

    That is fucking UTF-16 encoding!

    The UTF-16 encoding for the "Right Single Quotation Mark"
    is hexadecimal 8217.

    The file has THOUSANDS of other possessive apostrophes encoded
    with ASCII 0x27. Why is this one encoded as UTF-16 0x8217?

    Take your "surrogate pairs" and stuff them up your
    clyster pipes, you fucking decrepit, retarded faggot.

    You dare to contradict me and I'll slam you into
    fucking oblivion. Fucking worthless worm scum.

    One punch with my fist into your ugly, bloated face
    will put you in your rightful place forever.

    It'll put an end to your egregious bullshit forever.

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From DFS@21:1/5 to Tyrone on Sat Dec 2 09:17:03 2023
    On 12/1/2023 11:58 AM, Tyrone wrote:

    I imported food.csv into SQLite and MS Access with no problems at all.

    If you want to auto-import all .csv files in a folder into Access tables:


    Dim db As Database
    Set db = CurrentDb()

    Dim csvfolder As String, fname As String, filename As String, tblname
    As String
    Dim i As Integer
    csvfolder = path to your csv files
    'csvfolder = "D:\computer\dev\datasets\USDA\food\2023\AllFoods\csv\"

    Dim startTime As Date, endTime As Date
    startTime = Timer

    i = 1
    fname = Dir(csvfolder, vbDirectory)
    While fname <> ""
    If Len(fname) > 4 Then
    If Right(fname, 3) = "csv" Then
    Debug.Print i & ". " & fname
    tblname = Left(fname, Len(fname) - 4)
    filename = csvfolder & fname
    DoCmd.TransferText acImportDelim, , tblname, filename, True
    i = i + 1
    End If
    End If
    fname = Dir()
    Wend

    endTime = Timer
    Debug.Print "Finished: " & Format((endTime - startTime) / 60, "0.0") &
    " minutes"


    but Access (2003 at least) will choke when the file size hits 2GB, which
    it will if you import all these FoodCentral files.

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)