• Construction of Skybuck's UFO Alert Tool

    From Skybuck Flying@21:1/5 to All on Tue Aug 9 18:44:31 2022
    Construction of Skybuck's UFO Alert Tool log:

    *****************************************************************************************************
    Day 1, 31 july 2022, Scouting if UBER H3 compiles/builds/works and can be used from Delphi, SUCCESS !
    *****************************************************************************************************

    Video log of my activities of Day 1:
    https://youtu.be/b_Dw0iX7DRg

    1.1 UBER H3 library works, it was build with VS 2019, CMAKE GUI was first used and later CMAKE command used to make DLL version.
    https://h3geo.org/
    https://h3geo.org/docs/api/indexing
    https://github.com/uber/h3

    1.2 Delphi's TDUMP was used to inspect the produced H3 dll library and exported functions.
    https://www.embarcadero.com/products/delphi

    1.3 VIRUS TOTAL was used to confirm the H3 dll is a PE32+ library which means it's 64 bit.
    https://www.virustotal.com/gui/

    (1.4 Lazarus/Free Pascal's H2PAS and H2PAS Wizard was tried (twice, different versions) to convert from C to PAS but FAILED to produce anything useable.)

    1.5 (Nes's) CHET was used to convert C api headers into Pascal headers some further work will be necessary but it already works and is usefull.
    https://github.com/neslib/Chet

    1.6 (Nes's) CLang for Delphi was used to build CHET successfully. https://github.com/neslib/Neslib.Clang

    1.7 UBER H3 library/api was loaded into Delphi console application and some simple functions tested OK.


    ************************************************************************************************************************************************
    Day 2, 1 august 2022, Implemented H3CellIndex in Delphi which will be superior to C/C++ thanks to Delphi's property language feature ! ;) =D :P*
    ************************************************************************************************************************************************

    Video log of my activities of Day 2:
    https://youtu.be/O0ce-hCjxZQ

    2.1 UBER H3CellIndex implemented in three different versions: 0.01 (fast), 0.02 (resolution range protected), 0.03 (cells integrated into one property)

    2.2 Test program written to test UBER H3CellIndex

    2.3 Investigated the ammount of pentagons and hexagons, the latter of which is huge, possibly special rendering library needed to handle it.
    https://h3geo.org/docs/core-library/restable

    2.4 Contacted https://github.com/stonexjr who has a nice geodesic grid rendering engine, it appears it does not yet use or support UBER H3, filled
    an issue/request/idea that maybe it might interested him to implement/integrate it (off/after video/live recording):
    https://github.com/stonexjr/geodesic_grid_dvr/issues/1



    ********************************************************************************************************************************************
    Day 3, 3 august 2022, Trying to understand H3 indexing/coordinate systems + Investigated possibility of rendering a sphere/earth with OpenGL
    ********************************************************************************************************************************************

    Video log of my activities of Day 3:
    https://youtu.be/nD6g7APLhMY

    3.0 Trying to understand how H3 indexing works, it has multiple coordinate systems, it could get quite complex, I don't yet fully grasp the coordinate
    systems, so this shall require some more studieing and experimenting.

    https://h3geo.org/docs/core-library/coordsystems/ https://h3geo.org/docs/core-library/geoToH3desc https://h3geo.org/docs/core-library/h3ToGeoDesc/ https://h3geo.org/docs/core-library/h3ToGeoBoundaryDesc

    Further clarifications about coordinate systems may be found here: https://github.com/uber/h3-js/issues/84

    3.1 Custom OpenGL was tried, by missing 3D perspective matrices.

    3.1.1 Experiment with threaded renderer/opengl, surprisingly it worked somewhat.

    First a custom opengl renderer was tried, but I more or less discovered that OpenGL does not seem to apply 3D effect automatically.
    To create the 3D effect some kind of perspective matrices must be computed/created which is very boring to do, so I decided to investigate GLScene which
    has already implemented all of this.

    3.1.2 Investigated the possibility of creating custom 3D perspective matrix, but just boring and waste of time for now, maybe in future.

    This man explains how to create those 3D perspective matrices as well as other youtube videos/links.
    Basically it's a sign to use GLScene and avoid all that work. However GLScene might throw some exceptions in the future and then maybe a custom renderer
    could be tried to avoid that those problems, or hack GLScene instead to fix those exceptions, so far GLScene seems promising.
    https://www.youtube.com/watch?v=LhQ85bPCAJ8
    as well as this:
    https://www.youtube.com/watch?v=U0_ONQQ5ZNM

    3.2 Switch to GLScene library which has lots of support classes/components for working with 3D scenes and OpenGL.

    3.2.1 Downloading GLScene (I downloaded and used the zip file (GLScene_v2.1_VCL_Win32_Win64) from sourceforge)
    http://glscene.sourceforge.net/wikka/ https://sourceforge.net/projects/glscene/files/latest/download https://github.com/GLScene

    3.2.3 Installing GLScene

    Following the installation guide/pdf allowed me to install GLScene flawlessly, it is not necessary to copy any DLLs into the windows system folder,
    I also highly recommend not to do this. These DLLs if used for for example sound and such should later be distributed with the application anyway.
    So it's good to be able to test what the effect would be if the DLLs are missing.
    The only real step required to install GLScene is to add the source folder to the library path inside Delphi and then open the GLScene.groupproj
    and choose compile all, then per BPL/DT (design time) install it into the Delphi environment, only GLScene_DT needs to be installed to get GLSCene/OpenGL working,
    the rest is optional for additional functionality. ..\GLScene\Help\InstallationGLS.pdf

    3.2.4 Compiling/building GLScene

    Delphi compiler is lightning fast and compiled GLScene pretty quickly, in about a minute or less. C/C++ would have taken forever ! LOL
    GLScene consists out of at least 2.7 million lines of code. The earth's executable size was still below 4 megabyte which for now for me at least is acceptable.
    The GR32 RT (runtime) and DT (design time) BPL did not compile due to missing code for GR32 which is a graphics32 library but this code is not necessary.
    Only GLScene RT and DT where installed for now.
    (
    Installing and compiling/building GR32/Graphics32 might speed up the application even further, but might also introduce some more complexity and potential for bugs,
    but could be tried, the source code was somewhat hard to find, but can be found here:
    https://github.com/graphics32/graphics32
    ).

    3.2.5 Testing GLScene thanks to quick start guide
    Following this very simple quick start guide proved that GLscene can work: https://www.ibrtses.com/opengl/glscene_quickstart.html

    3.2.6 I came across a "Earth" advanced demo which could be perfect for Skybuck's UFO Alert Tool.

    While browsing the GLSCene website and it's demos/examples I came across this "Earth" advance demo which is awesome it even has sky light scattering.

    ..\GLScene\AdvDemos\Earth

    GLScene (version GLScene_v2.1_VCL_Win32_Win64) has a very good advanced demo of Earth and it should be just fine for Skybuck's UFO Alert Tool.
    And GLScene is necessary to get some "3D" look, otherwise it would be necessary to implement perspective matrices which is very boring for me to do.
    The software rendering mode was tried and does work, it is possible to set it. Though the earth advance demo does produce some integer overflow and
    range check errors in software mode. The advanced earth demo renders some sky/light scattering thingy, not sure if this is the cause of it.
    Most likely the real cause is some kind of texturing issue and mipmapping issue that might not work so well in software rendering mode.
    (I report this issue on github and also on the sourceforge page of GLScene, maybe somebody will fix it, but I doubt it for now.)

    Most likely it is possible to re-create this earth demo step by step to see how far it will work in software rendering mode.

    (Perhaps an option can be added to switch to software rendering mode for the user, however this should be in some ini file/text based so user can change
    it in case of problems starting up the application and/or the software should automatically fall back to software rendering mode in case of problems
    however GLScene already claims to do that so should be ok. In my case the GPU is half broken and it did not automatically switch to software rendering mode,
    which in my case allowed me to at least run the demo to get some kind of impression on this broken GPU.)

    Discovering this advanced demo of Earth and the usage of GLScene will most likely seriously reduce my development time for this UFO Alert Tool.
    That is the power of Delphi ! =D Quality code as your fingers tips ! ;) =D

    Maybe this will be helpfull in the future perhaps for routing network to figure some things out, for now I include it in this log:
    https://stackoverflow.com/questions/7705228/hexagonal-grids-how-do-you-find-which-hexagon-a-point-is-in

    Not sure if this will be usefull, maybe:
    HexASCII: a file format for cartographical hexagonal rasters file:///C:/Users/new/Downloads/HASC.pdf

    *New* Construction of Skybuck's UFO Alert Tool Playlist: https://youtube.com/playlist?list=PL0HGds8aHQsB47NC9ESZrG5r8WUvg8gU6


    *******************************************************************************************************************************************
    Day 4, 5 august 2022, Fixed advanced earth demo for software rendering. Massive fails: Tried to download/load better 3D Earth + Google maps
    *******************************************************************************************************************************************

    Video log of my activities of Day 4:
    https://youtu.be/pIKpmfasAec

    4.0 Even though day 4 ended in a massive fail, it was still an important day. This day I discovered how to fix the software rendering for GLScene.

    This is especially handy for the advanced demo: Earth.
    In folder:
    ..\GLScene\AdvDemos\Earth

    4.1 Fixing the Advanced Earth Demo for software rendering:

    4.1.1. Change GLSceneViewer.Buffer.ContextOptions.roSoftwareMode to true.

    4.1.2. Change GLSceneViewer.Buffer.AccumBufferBits from 0 to 8, 16, 24 or 32.

    4.1.3. Deleting the flare component from the sun component:

    GLScene(right click)->Show Scene Editor->LSSun->GLLensFlare1

    Right click->delete object

    4.1.4. Disable following two lines in GLScene library unit: GLS.Graphics.pas

    for slice := 0 to d - 1 do
    begin
    // Skybuck: disabled, buggy in software mode
    // Build2DMipmap(GetLevelAddress(0), LAddresses, fColorFormat, fDataType, // AFilter, GetWidth, GetHeight);
    for level := 1 to fLevelCount - 1 do
    Inc(PByte(LAddresses[level - 1]), GetLevelSizeInByte(level) div d);
    end;

    4.1.5. Disable the following line in fEarthd.pas:

    procedure TForm1.GLSceneViewerBeforeRender(Sender: TObject);
    begin
    // GLLensFlare1.PreRender(Sender as TGLSceneBuffer);

    4.2 A bigger/more detailed Earth 3D model was tried. However it sucked, the 3D model itself was just a sphere with triangles the texture maps
    where weird. Most 3D Model sites found with google suck and require payment of money.
    https://sketchfab.com/3d-models/earth-globe-98d2b04d46474bafb4250cc75dc583b3

    4.2.1 Conversion from FBX to OBJ was done: https://www.autodesk.com/developer-network/platform-technologies/fbx-converter-archives

    4.2.2 Difficulty loading MTL into GLScene, FAILED, undocumented, unknown.

    4.2.3 Difficulty loading Texture maps into GLScene, not even tried, because of 4.2.2

    4.3 Google Earth library 1.5.3 for Delphi was tried. However it cannot be used because Delphi's TWebBrowser component relies on Internet Explorer and
    Internet Explorer does not support the javascript api of google earth. Very maybe chromium browser could be used by changing some include file.
    Howebver chromium is a different browser than chrome from google. Chromium requires some kind of building of files, way too much work for nothing most likely
    so this was not tried.
    https://sourceforge.net/projects/gmlibrary/


    ***************************************************************************************************************************************************
    Day 5, 6 august 2022, Loading JSON file with country coordinates/lines and visualized/converted them to X,Y,Z coordinates on globe, quite awesome !
    ***************************************************************************************************************************************************

    (Forgot to post this link in day 3 log so for those that missed it here it is): Video log of my activities of Day 3:
    https://youtu.be/nD6g7APLhMY

    Video log of my activities of Day 5:
    https://youtu.be/OVv7J6YfM0o

    5.1 This day was an awesome day, it made me so happy I played the starwars theme song as I completed the mission for this day:

    Load the country borders of earth, try to visualize them in GLScene and added bonus, try and convert the coordinates from latitude, longitude to X,Y,Z
    3d coordinates on a sphere. Big Success ! JSON was parsed using a library I already had installed and used before:
    For easy to use, very powerfull, very fast JSON parser !:

    5.1.1 JSON data download for earth country lines: https://github.com/martynafford/natural-earth-geojson

    5.1.2 JSON Parser used:
    https://github.com/ahausladen/JsonDataObjects

    5.1.3 Explanation of Earth X,Y,Z coordinate system: https://en.wikipedia.org/wiki/Earth-centered,_Earth-fixed_coordinate_system

    5.1.4. Explanation of lat/long to x,y,z conversion formula: https://en.wikipedia.org/wiki/Geographic_coordinate_conversion

    5.1.5 Converted this python code to Delphi to convert Latitude, Longitude coordinates to X,Y,Z:
    https://gist.github.com/mpkuse/d7e2f29952b507921e3da2d7a26d1d93

    5.2 Scaled Earth in Y direction (important to visualize lines better).

    I also played a bit with earth scaling options, this is kinda cool, now earth can be a bit more realistic by scaling it in the Y direction slightly,
    this also helps to render the lines for the countries better.

    5.3 Playing with atmosphere.

    I also played a bit with the atmospheric rendering/constants to try and make it look a bit better, using real earth radius and such in km:
    https://en.wikipedia.org/wiki/Earth_radius

    5.4 High detail of country lines (10m a little bit slow on software rendering)

    I also managed to load a more high detail version of the lines to show the country lines better, this does slow down the renderer a little bit, a slightly
    less accurate version could also be tried. So far the 10m seems to be the most accurate. A 50m version could be tried to speed it up a little bit.
    (Same link as above, mentioned one more time here ;)) https://github.com/martynafford/natural-earth-geojson

    5.5.1 Huge Visible Earth Map (failed to load, too big)

    I then proceed to try and download and load a VisibleEarth High Resolution Map (43200x21600) assuming it would simply display and load in glscene.
    https://www.h-schmidt.net/map/

    5.5.2 Windows 7 System Crash, and Image/Photo/Graphics Software Failures

    Boy was I in for a surprise: Not only did windows 7 completely crash and restart, GLScene/Delphi also fails to load it because TBitmap cannot handle this size,
    All other software I tried also failed: paint, mspaint.net, glassimage, gimp and photoviewer. All for different reasons, mostly out of memory reasons or 3D issues.

    So unfortunately the day ended in another mass fail, but good progress was made ! ;)
    (I didn't see the crash of windows 7 because I bailed to my bed to rest lol, but when I came back the system had rebooted and I checked the system log):
    Bugcheck: 0x00000018 (0x0000000000000000, 0xfffffa8005fe0570, 0x0000000000000002, 0xffffffffffffffff)

    There were some other peculiar thing in the system log, like the firmware limitting certain processors, possibly system getting to hot.
    Main cause of crash seems to be failure of system to respond in time, possibly caused by out of memory situation.
    I checked the youtube stream nothing is visible on it concerning the crash lol. It would be diserably to have a seperate system for live streaming
    to catch these crashes lol, not that it matters that much, just a funny thing to see/notice, but could also be used to attack windows systems !


    *************************************************
    Day 6, 9 august 2022, Provinces and Hexagon World *************************************************

    Video log of my activities of Day 6:
    https://youtu.be/dEiZ3wIyaLw

    6.1. Provinces was tried/implemented, however there is a backface culling issues with the input data (version 4).
    I have found the original source which might have a better version, version 5, issues also reported on multiple sites, however the original source
    uses a different format, no JSON is available, but dbf is available dbase database file format which can be handled by Delphi.
    Original source of data seems to be: https://github.com/nvkelso/natural-earth-vector

    6.2. Hexagon world implemented by using the UBER H3 library. It isn't completely perfect yet, but already looks pretty good. Still trying to wrap my head around it.

    To solve the problem with the provinces two hexagon worlds could be created. One for zooming, and one which describes the "ground" of earth and it's provinces
    and/or countries. This could be done with UBER H3 compression technique where large parts are covered with large hexagons and smaller ones with little ones.
    However for routing purposes it would probably be better to main the same size, but for visualization purposes this compression might help.

    Possibilities for further research is either zooming functionality where level of detail is adjusted as user zoom in closer. Or a selection technique
    where only the selected hexagon to user is pointing towards is zoomed in... which would be a cool/funny technique maybe ! ;)

    Now that hexagon grid is working, more time should be spent on the networking and gui design of ufo alert tool and the routing of messages and/or broadcast
    messages to alert people of the presence of a ufo, especially if within their region. Plus perhaps also some time on some kind of server based/algorithm
    to distribute the initial peers to the main hexagons or so and perhaps further research into "leader" selection per (main) hexagon or so...

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From a a@21:1/5 to Skybuck Flying on Wed Aug 10 02:49:42 2022
    On Wednesday, 10 August 2022 at 03:44:36 UTC+2, Skybuck Flying wrote:
    Construction of Skybuck's UFO Alert Tool log:

    *****************************************************************************************************
    Day 1, 31 july 2022, Scouting if UBER H3 compiles/builds/works and can be used from Delphi, SUCCESS !
    *****************************************************************************************************

    Video log of my activities of Day 1:
    https://youtu.be/b_Dw0iX7DRg

    1.1 UBER H3 library works, it was build with VS 2019, CMAKE GUI was first used and later CMAKE command used to make DLL version.
    https://h3geo.org/
    https://h3geo.org/docs/api/indexing
    https://github.com/uber/h3

    1.2 Delphi's TDUMP was used to inspect the produced H3 dll library and exported functions.
    https://www.embarcadero.com/products/delphi

    1.3 VIRUS TOTAL was used to confirm the H3 dll is a PE32+ library which means it's 64 bit.
    https://www.virustotal.com/gui/

    (1.4 Lazarus/Free Pascal's H2PAS and H2PAS Wizard was tried (twice, different versions) to convert from C to PAS but FAILED to produce anything useable.)

    1.5 (Nes's) CHET was used to convert C api headers into Pascal headers some further work will be necessary but it already works and is usefull.
    https://github.com/neslib/Chet

    1.6 (Nes's) CLang for Delphi was used to build CHET successfully. https://github.com/neslib/Neslib.Clang

    1.7 UBER H3 library/api was loaded into Delphi console application and some simple functions tested OK.


    ************************************************************************************************************************************************
    Day 2, 1 august 2022, Implemented H3CellIndex in Delphi which will be superior to C/C++ thanks to Delphi's property language feature ! ;) =D :P*
    ************************************************************************************************************************************************

    Video log of my activities of Day 2:
    https://youtu.be/O0ce-hCjxZQ

    2.1 UBER H3CellIndex implemented in three different versions: 0.01 (fast), 0.02 (resolution range protected), 0.03 (cells integrated into one property)

    2.2 Test program written to test UBER H3CellIndex

    2.3 Investigated the ammount of pentagons and hexagons, the latter of which is huge, possibly special rendering library needed to handle it.
    https://h3geo.org/docs/core-library/restable

    2.4 Contacted https://github.com/stonexjr who has a nice geodesic grid rendering engine, it appears it does not yet use or support UBER H3, filled
    an issue/request/idea that maybe it might interested him to implement/integrate it (off/after video/live recording):
    https://github.com/stonexjr/geodesic_grid_dvr/issues/1



    ********************************************************************************************************************************************
    Day 3, 3 august 2022, Trying to understand H3 indexing/coordinate systems + Investigated possibility of rendering a sphere/earth with OpenGL
    ********************************************************************************************************************************************

    Video log of my activities of Day 3:
    https://youtu.be/nD6g7APLhMY

    3.0 Trying to understand how H3 indexing works, it has multiple coordinate systems, it could get quite complex, I don't yet fully grasp the coordinate
    systems, so this shall require some more studieing and experimenting.

    https://h3geo.org/docs/core-library/coordsystems/ https://h3geo.org/docs/core-library/geoToH3desc https://h3geo.org/docs/core-library/h3ToGeoDesc/ https://h3geo.org/docs/core-library/h3ToGeoBoundaryDesc

    Further clarifications about coordinate systems may be found here: https://github.com/uber/h3-js/issues/84

    3.1 Custom OpenGL was tried, by missing 3D perspective matrices.

    3.1.1 Experiment with threaded renderer/opengl, surprisingly it worked somewhat.

    First a custom opengl renderer was tried, but I more or less discovered that OpenGL does not seem to apply 3D effect automatically.
    To create the 3D effect some kind of perspective matrices must be computed/created which is very boring to do, so I decided to investigate GLScene which
    has already implemented all of this.

    3.1.2 Investigated the possibility of creating custom 3D perspective matrix, but just boring and waste of time for now, maybe in future.

    This man explains how to create those 3D perspective matrices as well as other youtube videos/links.
    Basically it's a sign to use GLScene and avoid all that work. However GLScene might throw some exceptions in the future and then maybe a custom renderer
    could be tried to avoid that those problems, or hack GLScene instead to fix those exceptions, so far GLScene seems promising.
    https://www.youtube.com/watch?v=LhQ85bPCAJ8
    as well as this:
    https://www.youtube.com/watch?v=U0_ONQQ5ZNM

    3.2 Switch to GLScene library which has lots of support classes/components for working with 3D scenes and OpenGL.

    3.2.1 Downloading GLScene (I downloaded and used the zip file (GLScene_v2.1_VCL_Win32_Win64) from sourceforge)
    http://glscene.sourceforge.net/wikka/ https://sourceforge.net/projects/glscene/files/latest/download https://github.com/GLScene

    3.2.3 Installing GLScene

    Following the installation guide/pdf allowed me to install GLScene flawlessly, it is not necessary to copy any DLLs into the windows system folder,
    I also highly recommend not to do this. These DLLs if used for for example sound and such should later be distributed with the application anyway.
    So it's good to be able to test what the effect would be if the DLLs are missing.
    The only real step required to install GLScene is to add the source folder to the library path inside Delphi and then open the GLScene.groupproj
    and choose compile all, then per BPL/DT (design time) install it into the Delphi environment, only GLScene_DT needs to be installed to get GLSCene/OpenGL working,
    the rest is optional for additional functionality. ..\GLScene\Help\InstallationGLS.pdf

    3.2.4 Compiling/building GLScene

    Delphi compiler is lightning fast and compiled GLScene pretty quickly, in about a minute or less. C/C++ would have taken forever ! LOL
    GLScene consists out of at least 2.7 million lines of code. The earth's executable size was still below 4 megabyte which for now for me at least is acceptable.
    The GR32 RT (runtime) and DT (design time) BPL did not compile due to missing code for GR32 which is a graphics32 library but this code is not necessary.
    Only GLScene RT and DT where installed for now.
    (
    Installing and compiling/building GR32/Graphics32 might speed up the application even further, but might also introduce some more complexity and potential for bugs,
    but could be tried, the source code was somewhat hard to find, but can be found here:
    https://github.com/graphics32/graphics32
    ).

    3.2.5 Testing GLScene thanks to quick start guide
    Following this very simple quick start guide proved that GLscene can work: https://www.ibrtses.com/opengl/glscene_quickstart.html

    3.2.6 I came across a "Earth" advanced demo which could be perfect for Skybuck's UFO Alert Tool.

    While browsing the GLSCene website and it's demos/examples I came across this "Earth" advance demo which is awesome it even has sky light scattering.

    ..\GLScene\AdvDemos\Earth

    GLScene (version GLScene_v2.1_VCL_Win32_Win64) has a very good advanced demo of Earth and it should be just fine for Skybuck's UFO Alert Tool.
    And GLScene is necessary to get some "3D" look, otherwise it would be necessary to implement perspective matrices which is very boring for me to do.
    The software rendering mode was tried and does work, it is possible to set it. Though the earth advance demo does produce some integer overflow and
    range check errors in software mode. The advanced earth demo renders some sky/light scattering thingy, not sure if this is the cause of it.
    Most likely the real cause is some kind of texturing issue and mipmapping issue that might not work so well in software rendering mode.
    (I report this issue on github and also on the sourceforge page of GLScene, maybe somebody will fix it, but I doubt it for now.)

    Most likely it is possible to re-create this earth demo step by step to see how far it will work in software rendering mode.

    (Perhaps an option can be added to switch to software rendering mode for the user, however this should be in some ini file/text based so user can change
    it in case of problems starting up the application and/or the software should automatically fall back to software rendering mode in case of problems
    however GLScene already claims to do that so should be ok. In my case the GPU is half broken and it did not automatically switch to software rendering mode,
    which in my case allowed me to at least run the demo to get some kind of impression on this broken GPU.)

    Discovering this advanced demo of Earth and the usage of GLScene will most likely seriously reduce my development time for this UFO Alert Tool.
    That is the power of Delphi ! =D Quality code as your fingers tips ! ;) =D

    Maybe this will be helpfull in the future perhaps for routing network to figure some things out, for now I include it in this log:
    https://stackoverflow.com/questions/7705228/hexagonal-grids-how-do-you-find-which-hexagon-a-point-is-in

    Not sure if this will be usefull, maybe:
    HexASCII: a file format for cartographical hexagonal rasters file:///C:/Users/new/Downloads/HASC.pdf

    *New* Construction of Skybuck's UFO Alert Tool Playlist: https://youtube.com/playlist?list=PL0HGds8aHQsB47NC9ESZrG5r8WUvg8gU6


    *******************************************************************************************************************************************
    Day 4, 5 august 2022, Fixed advanced earth demo for software rendering. Massive fails: Tried to download/load better 3D Earth + Google maps
    *******************************************************************************************************************************************

    Video log of my activities of Day 4:
    https://youtu.be/pIKpmfasAec

    4.0 Even though day 4 ended in a massive fail, it was still an important day. This day I discovered how to fix the software rendering for GLScene.

    This is especially handy for the advanced demo: Earth.
    In folder:
    ..\GLScene\AdvDemos\Earth

    4.1 Fixing the Advanced Earth Demo for software rendering:

    4.1.1. Change GLSceneViewer.Buffer.ContextOptions.roSoftwareMode to true.

    4.1.2. Change GLSceneViewer.Buffer.AccumBufferBits from 0 to 8, 16, 24 or 32.

    4.1.3. Deleting the flare component from the sun component:

    GLScene(right click)->Show Scene Editor->LSSun->GLLensFlare1

    Right click->delete object

    4.1.4. Disable following two lines in GLScene library unit: GLS.Graphics.pas

    for slice := 0 to d - 1 do
    begin
    // Skybuck: disabled, buggy in software mode
    // Build2DMipmap(GetLevelAddress(0), LAddresses, fColorFormat, fDataType,
    // AFilter, GetWidth, GetHeight);
    for level := 1 to fLevelCount - 1 do
    Inc(PByte(LAddresses[level - 1]), GetLevelSizeInByte(level) div d);
    end;

    4.1.5. Disable the following line in fEarthd.pas:

    procedure TForm1.GLSceneViewerBeforeRender(Sender: TObject);
    begin
    // GLLensFlare1.PreRender(Sender as TGLSceneBuffer);

    4.2 A bigger/more detailed Earth 3D model was tried. However it sucked, the 3D model itself was just a sphere with triangles the texture maps
    where weird. Most 3D Model sites found with google suck and require payment of money.
    https://sketchfab.com/3d-models/earth-globe-98d2b04d46474bafb4250cc75dc583b3

    4.2.1 Conversion from FBX to OBJ was done: https://www.autodesk.com/developer-network/platform-technologies/fbx-converter-archives

    4.2.2 Difficulty loading MTL into GLScene, FAILED, undocumented, unknown.

    4.2.3 Difficulty loading Texture maps into GLScene, not even tried, because of 4.2.2

    4.3 Google Earth library 1.5.3 for Delphi was tried. However it cannot be used because Delphi's TWebBrowser component relies on Internet Explorer and
    Internet Explorer does not support the javascript api of google earth. Very maybe chromium browser could be used by changing some include file.
    Howebver chromium is a different browser than chrome from google. Chromium requires some kind of building of files, way too much work for nothing most likely
    so this was not tried.
    https://sourceforge.net/projects/gmlibrary/


    ***************************************************************************************************************************************************
    Day 5, 6 august 2022, Loading JSON file with country coordinates/lines and visualized/converted them to X,Y,Z coordinates on globe, quite awesome !
    ***************************************************************************************************************************************************

    (Forgot to post this link in day 3 log so for those that missed it here it is):
    Video log of my activities of Day 3:
    https://youtu.be/nD6g7APLhMY

    Video log of my activities of Day 5:
    https://youtu.be/OVv7J6YfM0o

    5.1 This day was an awesome day, it made me so happy I played the starwars theme song as I completed the mission for this day:

    Load the country borders of earth, try to visualize them in GLScene and added bonus, try and convert the coordinates from latitude, longitude to X,Y,Z
    3d coordinates on a sphere. Big Success ! JSON was parsed using a library I already had installed and used before:
    For easy to use, very powerfull, very fast JSON parser !:

    5.1.1 JSON data download for earth country lines: https://github.com/martynafford/natural-earth-geojson

    5.1.2 JSON Parser used:
    https://github.com/ahausladen/JsonDataObjects

    5.1.3 Explanation of Earth X,Y,Z coordinate system: https://en.wikipedia.org/wiki/Earth-centered,_Earth-fixed_coordinate_system

    5.1.4. Explanation of lat/long to x,y,z conversion formula: https://en.wikipedia.org/wiki/Geographic_coordinate_conversion

    5.1.5 Converted this python code to Delphi to convert Latitude, Longitude coordinates to X,Y,Z:
    https://gist.github.com/mpkuse/d7e2f29952b507921e3da2d7a26d1d93

    5.2 Scaled Earth in Y direction (important to visualize lines better).

    I also played a bit with earth scaling options, this is kinda cool, now earth can be a bit more realistic by scaling it in the Y direction slightly,
    this also helps to render the lines for the countries better.

    5.3 Playing with atmosphere.

    I also played a bit with the atmospheric rendering/constants to try and make it look a bit better, using real earth radius and such in km:
    https://en.wikipedia.org/wiki/Earth_radius

    5.4 High detail of country lines (10m a little bit slow on software rendering)

    I also managed to load a more high detail version of the lines to show the country lines better, this does slow down the renderer a little bit, a slightly
    less accurate version could also be tried. So far the 10m seems to be the most accurate. A 50m version could be tried to speed it up a little bit.
    (Same link as above, mentioned one more time here ;)) https://github.com/martynafford/natural-earth-geojson

    5.5.1 Huge Visible Earth Map (failed to load, too big)

    I then proceed to try and download and load a VisibleEarth High Resolution Map (43200x21600) assuming it would simply display and load in glscene.
    https://www.h-schmidt.net/map/

    5.5.2 Windows 7 System Crash, and Image/Photo/Graphics Software Failures

    Boy was I in for a surprise: Not only did windows 7 completely crash and restart, GLScene/Delphi also fails to load it because TBitmap cannot handle this size,
    All other software I tried also failed: paint, mspaint.net, glassimage, gimp and photoviewer. All for different reasons, mostly out of memory reasons or 3D issues.

    So unfortunately the day ended in another mass fail, but good progress was made ! ;)
    (I didn't see the crash of windows 7 because I bailed to my bed to rest lol, but when I came back the system had rebooted and I checked the system log):
    Bugcheck: 0x00000018 (0x0000000000000000, 0xfffffa8005fe0570, 0x0000000000000002, 0xffffffffffffffff)

    There were some other peculiar thing in the system log, like the firmware limitting certain processors, possibly system getting to hot.
    Main cause of crash seems to be failure of system to respond in time, possibly caused by out of memory situation.
    I checked the youtube stream nothing is visible on it concerning the crash lol. It would be diserably to have a seperate system for live streaming
    to catch these crashes lol, not that it matters that much, just a funny thing to see/notice, but could also be used to attack windows systems !


    *************************************************
    Day 6, 9 august 2022, Provinces and Hexagon World *************************************************

    Video log of my activities of Day 6:
    https://youtu.be/dEiZ3wIyaLw

    6.1. Provinces was tried/implemented, however there is a backface culling issues with the input data (version 4).
    I have found the original source which might have a better version, version 5, issues also reported on multiple sites, however the original source
    uses a different format, no JSON is available, but dbf is available dbase database file format which can be handled by Delphi.
    Original source of data seems to be: https://github.com/nvkelso/natural-earth-vector

    6.2. Hexagon world implemented by using the UBER H3 library. It isn't completely perfect yet, but already looks pretty good. Still trying to wrap my head around it.

    To solve the problem with the provinces two hexagon worlds could be created. One for zooming, and one which describes the "ground" of earth and it's provinces
    and/or countries. This could be done with UBER H3 compression technique where large parts are covered with large hexagons and smaller ones with little ones.
    However for routing purposes it would probably be better to main the same size, but for visualization purposes this compression might help.

    Possibilities for further research is either zooming functionality where level of detail is adjusted as user zoom in closer. Or a selection technique
    where only the selected hexagon to user is pointing towards is zoomed in... which would be a cool/funny technique maybe ! ;)

    Now that hexagon grid is working, more time should be spent on the networking and gui design of ufo alert tool and the routing of messages and/or broadcast
    messages to alert people of the presence of a ufo, especially if within their region. Plus perhaps also some time on some kind of server based/algorithm
    to distribute the initial peers to the main hexagons or so and perhaps further research into "leader" selection per (main) hexagon or so...

    read
    My Life Bits by Mic rosoft


    MyLifeBits - Microsoft Research https://www.microsoft.com/en-us/research/project/mylifebits

    02.11.2001 · MylifeBits is a lifetime store of everything. It is the fulfillment of Vannevar Bush’s 1945 Memex vision including full-text search, text & audio annotations, and hyperlinks. The book Total Recall (paperback title Your Life, Uploaded) is
    the culmination of our thoughts regarding MyLifebits and the larger CARPE research agenda. There are two parts to MyLifeBits: an …


    MyLifeBits - Wikipedia
    https://en.wikipedia.org/wiki/MyLifeBits

    Overview
    See also
    External links

    MyLifeBits is a life-logging experiment begun in 2001. It is a Microsoft Research project inspired by Vannevar Bush's hypothetical Memex computer system. The project includes full-text search, text and audio annotations, and hyperlinks. The "experimental
    subject" of the project is computer scientist Gordon Bell, and the project will try to collect a lifetime of storage on and about Bell. Jim Gemmell of Microsoft Research and Roger Lueder were the architects and creators of the syste…
    Wikipedia · CC-BY-SA



    MyLifeBits: a personal database for everything - Microsoft Research https://www.microsoft.com/en-us/research/...

    01.01.2006 · MyLifeBits is a system that began in 2001 to explore the use of SQL to store all personal information found in PCs. The system initially focused on capturing and storing scanned and encoded archival material e.g. articles, books, music,
    photos, and video as well as everything born digital e.g. office documents, email, digital photos.

    Autor: Jim Gemmell, Gordon Bell, Roger Lueder


    Publish Year: 2006

    MyLifeBits | Microsoft Wiki | Fandom https://microsoft.fandom.com/wiki/MyLifeBits

    Mylifebits
    The Experimental Side
    Software Research

    MyLifeBits is a system that allows you to store the events of your life in the PC so that you can always review what you once saw, or show it to friends and family members. The project is the embodiment of the hypothetical computer system Memex
    Vanivar Bush. The project includes full-text search, text and sound annotations, hyperlinks. The MyLifeBi...


    ===

    MyLifeBits
    Microsoft
    MyLifeBits is a project developed by Microsoft Research, the Microsoft Search Division, which consists in recording and managing all the events of a person's life through data and multimedia platforms. MyLifeBits is a system that allows you to store the
    events of your life in the PC so that you can always review what you on… MyLifeBits | Microsoft Wiki | Fandom https://microsoft.fandom.com/wiki/MyLifeBits
    MyLifeBits
    MyLifeBits is a system that allows you to store the events of your life in the PC so that you can always review what you once saw, or show it to friends and family members. The project is the embodiment of the hypothetical computer system Memex Vanivar
    Bush. The project includes full-text search, text and sound annotations, hyperlinks. The MyLifeBits project consists of two parts: an experiment in the storage of lifelong information and research in the field of software. Czytaj więcej
    The experimental side
    During the MyLifeBits experiment, all electronic interactions of an individual (phone calls, e-mail, documents) are stored electronically. The collected data includes minute pictures. The experimental side of the project was handled by Gordon Bell (
    Gordon Bell). He captured articles, books, postcards, CDs, letters, notes, documents, photographs, presentations, home movies, videotapes, voice recordings and kept them digitally for life. He wanted to keep digital phone calls, television and radio...

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From Skybuck Flying@21:1/5 to All on Thu Aug 18 19:57:00 2022
    ************************************************
    Day 6.5, 13 august 2022, Studied hexagon routing ************************************************

    (No video log available, too hot and I wanted to concentrate intensely and keep hexagon routing designs private for now until more confident it's good.)

    6.5.1 I studied document "An_Optimal_Permutation_Routing_Algorithm_on_Full-D.pdf" in quiet without live streaming, also because it was a hot day.

    6.5.2 I believe the algorithm can be extended with an additional axis to go up and down the uber h3 hexagon world resolutions.

    6.5.3 I did some initial code sketches for data structures and routines and such.

    6.5.4 It would be possible to adjust packet coordinates every time it moves, so that it would seem that each node is the center of the world to prevent/avoid
    wrap around problems with big coordinates suddenly going zero, testing will have to prove if this can work and if it works well.

    6.5.5 I believe this algorithm described in this document will work and is doable.

    ************************************
    Day 7, 15 august 2022, Further plans
    ************************************

    Video log of my activities of Day 7:
    https://youtu.be/WCw24JWUQ84

    Now that the previous objectives have more or less been met it's time to formulate new plans.

    This day and the days around it it was very very very hot so I kept a low profile, not trying to sweat to much, however these plans came out quite nicely.

    I went pretty deep with these plans but that's a good thing, that might make the tool awesome in the end, goals to strive for ! ;)

    I will briefly sum up the new plans/goals in this log here:

    7.1 ROUTING/BROADCAST
    7.2 MOUSE CURSOR, SELECTING HEXAGONS, SUB HEXAGONS, ZOOM IN/OUT
    7.3 ALLOW USER TO SETUP BASE OF OPERATION, ALSO FOR QUICK UFO ALERTING
    7.3.2 VOICE CONTROL
    7.4 CUSTOM HEXAGON SELECTION FOR UFO ALERTS (IN CASE MOBILE/MOVING OR OTHER PURPOSES, HEARD IT IN THE NEWS/FROM OTHERS THAT DONT HAVE THE APP/MAYBE TELEPHONE)
    7.5 MAIN SERVER, HTTP SERVER, PHP SCRIPTS, MYSQL DATABASE
    7.6 CELL ORGANIZATION/HIERARCHY
    7.7 BROADCAST PACKETS AND FORWARDING/ROUTING/RULES/SPAM PROTECTION
    7.8 TRAY ICON APPLICATION, HIDE & STAY RESIDENT/OPERATING
    7.9 VISUALIZING THE HEXAGON WORLD AND POTENTIALLY A MORE COMPLEX (ADDRESSABLE/INDEXABLE) HEXAGON DATA STRUCTURE
    7.10 VISUALIZE HEXAGON COORDINATES (/ COORDINATE SYSTEM, MAYBE ALSO AXIS COULD BE USEFULL, FOR PACKET DIRECTION TRAVEL)
    7.11 Visualizing/triangulatization of the hexagons, connecting the centers to form triangles
    7.12 MAIN MENU, VISUALIZATION OPTIONS
    7.13 Auto-update feature
    7.14 OPTIONS TO SPECIFY UFO ALERT TYPE
    7.15 MULTICAST GROUPS FOR GLOBAL COMMUNICATION AND DATA COLLECTION
    7.17 FILTERING UFO ALERT TYPES
    7.18 CONFIRMATIONS OF UFO ALERTS.
    7.19 LICENSE, DISCLAIMER, WAIVER MUST BE INCLUDED WITH SOFTWARE THAT PROTECTS DEVELOPER FROM CLAIMS/LAWSUITS

    ***************************************************************************** Day 8, 18 august 2022, Selecting hexagons, Zoom experiments, Text experiments *****************************************************************************

    Video log of my activities of Day 8:
    https://youtu.be/kyLGbveJl9o

    Still very hot, also sleeping is hurt somewhat by worrieing about aliens being real =D,
    so I am taking it easy, plus watching ufo related material is fascinating and I want to stay current/up to date :)

    8.1 Biggest achievement was being able to select hexagons this was actually quite easily achieveable thanks to great design of GLScene
    it's not perfect yet but needs a bit more work to not confuse the mouse button with rotation and such.

    8.2 Zoom experiments have been done, also with earth coordinates in the millions now. Zooming will need some further work too.

    8.3 Text experiments performed... will also need some further work, like rotate text so it's nicely aligned with cell.

    8.4 A possible problem discovered with Uber H3 library. It seems some APIs might be missing/not implemented during build. Not sure why,
    github issue was filed about this. If not solved an entire port of Uber H3 to Delphi might be necessary ! ;)

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)