********************************************************************************************************************************************
Day 3, 3 august 2022, Trying to understand H3 indexing/coordinate systems + Investigated possibility of rendering a sphere/earth with OpenGL
********************************************************************************************************************************************
Video log of my activities of Day 3:
https://youtu.be/nD6g7APLhMY
3.0 Trying to understand how H3 indexing works, it has multiple coordinate systems, it could get quite complex, I don't yet fully grasp the coordinate
systems, so this shall require some more studieing and experimenting.
https://h3geo.org/docs/core-library/coordsystems/ https://h3geo.org/docs/core-library/geoToH3desc https://h3geo.org/docs/core-library/h3ToGeoDesc/ https://h3geo.org/docs/core-library/h3ToGeoBoundaryDesc
Further clarifications about coordinate systems may be found here:
https://github.com/uber/h3-js/issues/84
3.1 Custom OpenGL was tried, by missing 3D perspective matrices.
3.1.1 Experiment with threaded renderer/opengl, surprisingly it worked somewhat.
First a custom opengl renderer was tried, but I more or less discovered that OpenGL does not seem to apply 3D effect automatically.
To create the 3D effect some kind of perspective matrices must be computed/created which is very boring to do, so I decided to investigate GLScene which
has already implemented all of this.
3.1.2 Investigated the possibility of creating custom 3D perspective matrix, but just boring and waste of time for now, maybe in future.
This man explains how to create those 3D perspective matrices as well as other youtube videos/links.
Basically it's a sign to use GLScene and avoid all that work. However GLScene might throw some exceptions in the future and then maybe a custom renderer
could be tried to avoid that those problems, or hack GLScene instead to fix those exceptions, so far GLScene seems promising.
https://www.youtube.com/watch?v=LhQ85bPCAJ8
as well as this:
https://www.youtube.com/watch?v=U0_ONQQ5ZNM
3.2 Switch to GLScene library which has lots of support classes/components for working with 3D scenes and OpenGL.
3.2.1 Downloading GLScene (I downloaded and used the zip file (GLScene_v2.1_VCL_Win32_Win64) from sourceforge)
http://glscene.sourceforge.net/wikka/ https://sourceforge.net/projects/glscene/files/latest/download https://github.com/GLScene
3.2.3 Installing GLScene
Following the installation guide/pdf allowed me to install GLScene flawlessly, it is not necessary to copy any DLLs into the windows system folder,
I also highly recommend not to do this. These DLLs if used for for example sound and such should later be distributed with the application anyway.
So it's good to be able to test what the effect would be if the DLLs are missing.
The only real step required to install GLScene is to add the source folder to the library path inside Delphi and then open the GLScene.groupproj
and choose compile all, then per BPL/DT (design time) install it into the Delphi environment, only GLScene_DT needs to be installed to get GLSCene/OpenGL working,
the rest is optional for additional functionality. ..\GLScene\Help\InstallationGLS.pdf
3.2.4 Compiling/building GLScene
Delphi compiler is lightning fast and compiled GLScene pretty quickly, in about a minute or less. C/C++ would have taken forever ! LOL
GLScene consists out of at least 2.7 million lines of code. The earth's executable size was still below 4 megabyte which for now for me at least is acceptable.
The GR32 RT (runtime) and DT (design time) BPL did not compile due to missing code for GR32 which is a graphics32 library but this code is not necessary.
Only GLScene RT and DT where installed for now.
(
Installing and compiling/building GR32/Graphics32 might speed up the application even further, but might also introduce some more complexity and potential for bugs,
but could be tried, the source code was somewhat hard to find, but can be found here:
https://github.com/graphics32/graphics32
).
3.2.5 Testing GLScene thanks to quick start guide
Following this very simple quick start guide proved that GLscene can work:
https://www.ibrtses.com/opengl/glscene_quickstart.html
3.2.6 I came across a "Earth" advanced demo which could be perfect for Skybuck's UFO Alert Tool.
While browsing the GLSCene website and it's demos/examples I came across this "Earth" advance demo which is awesome it even has sky light scattering.
..\GLScene\AdvDemos\Earth
GLScene (version GLScene_v2.1_VCL_Win32_Win64) has a very good advanced demo of Earth and it should be just fine for Skybuck's UFO Alert Tool.
And GLScene is necessary to get some "3D" look, otherwise it would be necessary to implement perspective matrices which is very boring for me to do.
The software rendering mode was tried and does work, it is possible to set it. Though the earth advance demo does produce some integer overflow and
range check errors in software mode. The advanced earth demo renders some sky/light scattering thingy, not sure if this is the cause of it.
Most likely the real cause is some kind of texturing issue and mipmapping issue that might not work so well in software rendering mode.
(I report this issue on github and also on the sourceforge page of GLScene, maybe somebody will fix it, but I doubt it for now.)
Most likely it is possible to re-create this earth demo step by step to see how far it will work in software rendering mode.
(Perhaps an option can be added to switch to software rendering mode for the user, however this should be in some ini file/text based so user can change
it in case of problems starting up the application and/or the software should automatically fall back to software rendering mode in case of problems
however GLScene already claims to do that so should be ok. In my case the GPU is half broken and it did not automatically switch to software rendering mode,
which in my case allowed me to at least run the demo to get some kind of impression on this broken GPU.)
Discovering this advanced demo of Earth and the usage of GLScene will most likely seriously reduce my development time for this UFO Alert Tool.
That is the power of Delphi ! =D Quality code as your fingers tips ! ;) =D
Maybe this will be helpfull in the future perhaps for routing network to figure some things out, for now I include it in this log:
https://stackoverflow.com/questions/7705228/hexagonal-grids-how-do-you-find-which-hexagon-a-point-is-in
Not sure if this will be usefull, maybe:
HexASCII: a file format for cartographical hexagonal rasters file:///C:/Users/new/Downloads/HASC.pdf
*New* Construction of Skybuck's UFO Alert Tool Playlist:
https://youtube.com/playlist?list=PL0HGds8aHQsB47NC9ESZrG5r8WUvg8gU6
*******************************************************************************************************************************************
Day 4, 5 august 2022, Fixed advanced earth demo for software rendering. Massive fails: Tried to download/load better 3D Earth + Google maps
*******************************************************************************************************************************************
Video log of my activities of Day 4:
https://youtu.be/pIKpmfasAec
4.0 Even though day 4 ended in a massive fail, it was still an important day. This day I discovered how to fix the software rendering for GLScene.
This is especially handy for the advanced demo: Earth.
In folder:
..\GLScene\AdvDemos\Earth
4.1 Fixing the Advanced Earth Demo for software rendering:
4.1.1. Change GLSceneViewer.Buffer.ContextOptions.roSoftwareMode to true.
4.1.2. Change GLSceneViewer.Buffer.AccumBufferBits from 0 to 8, 16, 24 or 32.
4.1.3. Deleting the flare component from the sun component:
GLScene(right click)->Show Scene Editor->LSSun->GLLensFlare1
Right click->delete object
4.1.4. Disable following two lines in GLScene library unit: GLS.Graphics.pas
for slice := 0 to d - 1 do
begin
// Skybuck: disabled, buggy in software mode
// Build2DMipmap(GetLevelAddress(0), LAddresses, fColorFormat, fDataType, // AFilter, GetWidth, GetHeight);
for level := 1 to fLevelCount - 1 do
Inc(PByte(LAddresses[level - 1]), GetLevelSizeInByte(level) div d);
end;
4.1.5. Disable the following line in fEarthd.pas:
procedure TForm1.GLSceneViewerBeforeRender(Sender: TObject);
begin
// GLLensFlare1.PreRender(Sender as TGLSceneBuffer);
4.2 A bigger/more detailed Earth 3D model was tried. However it sucked, the 3D model itself was just a sphere with triangles the texture maps
where weird. Most 3D Model sites found with google suck and require payment of money.
https://sketchfab.com/3d-models/earth-globe-98d2b04d46474bafb4250cc75dc583b3
4.2.1 Conversion from FBX to OBJ was done:
https://www.autodesk.com/developer-network/platform-technologies/fbx-converter-archives
4.2.2 Difficulty loading MTL into GLScene, FAILED, undocumented, unknown.
4.2.3 Difficulty loading Texture maps into GLScene, not even tried, because of 4.2.2
4.3 Google Earth library 1.5.3 for Delphi was tried. However it cannot be used because Delphi's TWebBrowser component relies on Internet Explorer and
Internet Explorer does not support the javascript api of google earth. Very maybe chromium browser could be used by changing some include file.
Howebver chromium is a different browser than chrome from google. Chromium requires some kind of building of files, way too much work for nothing most likely
so this was not tried.
https://sourceforge.net/projects/gmlibrary/
***************************************************************************************************************************************************
Day 5, 6 august 2022, Loading JSON file with country coordinates/lines and visualized/converted them to X,Y,Z coordinates on globe, quite awesome !
***************************************************************************************************************************************************
(Forgot to post this link in day 3 log so for those that missed it here it is): Video log of my activities of Day 3:
https://youtu.be/nD6g7APLhMY
Video log of my activities of Day 5:
https://youtu.be/OVv7J6YfM0o
5.1 This day was an awesome day, it made me so happy I played the starwars theme song as I completed the mission for this day:
Load the country borders of earth, try to visualize them in GLScene and added bonus, try and convert the coordinates from latitude, longitude to X,Y,Z
3d coordinates on a sphere. Big Success ! JSON was parsed using a library I already had installed and used before:
For easy to use, very powerfull, very fast JSON parser !:
5.1.1 JSON data download for earth country lines:
https://github.com/martynafford/natural-earth-geojson
5.1.2 JSON Parser used:
https://github.com/ahausladen/JsonDataObjects
5.1.3 Explanation of Earth X,Y,Z coordinate system:
https://en.wikipedia.org/wiki/Earth-centered,_Earth-fixed_coordinate_system
5.1.4. Explanation of lat/long to x,y,z conversion formula:
https://en.wikipedia.org/wiki/Geographic_coordinate_conversion
5.1.5 Converted this python code to Delphi to convert Latitude, Longitude coordinates to X,Y,Z:
https://gist.github.com/mpkuse/d7e2f29952b507921e3da2d7a26d1d93
5.2 Scaled Earth in Y direction (important to visualize lines better).
I also played a bit with earth scaling options, this is kinda cool, now earth can be a bit more realistic by scaling it in the Y direction slightly,
this also helps to render the lines for the countries better.
5.3 Playing with atmosphere.
I also played a bit with the atmospheric rendering/constants to try and make it look a bit better, using real earth radius and such in km:
https://en.wikipedia.org/wiki/Earth_radius
5.4 High detail of country lines (10m a little bit slow on software rendering)
I also managed to load a more high detail version of the lines to show the country lines better, this does slow down the renderer a little bit, a slightly
less accurate version could also be tried. So far the 10m seems to be the most accurate. A 50m version could be tried to speed it up a little bit.
(Same link as above, mentioned one more time here ;))
https://github.com/martynafford/natural-earth-geojson
5.5.1 Huge Visible Earth Map (failed to load, too big)
I then proceed to try and download and load a VisibleEarth High Resolution Map (43200x21600) assuming it would simply display and load in glscene.
https://www.h-schmidt.net/map/
5.5.2 Windows 7 System Crash, and Image/Photo/Graphics Software Failures
Boy was I in for a surprise: Not only did windows 7 completely crash and restart, GLScene/Delphi also fails to load it because TBitmap cannot handle this size,
All other software I tried also failed: paint, mspaint.net, glassimage, gimp and photoviewer. All for different reasons, mostly out of memory reasons or 3D issues.
So unfortunately the day ended in another mass fail, but good progress was made ! ;)
(I didn't see the crash of windows 7 because I bailed to my bed to rest lol, but when I came back the system had rebooted and I checked the system log):
Bugcheck: 0x00000018 (0x0000000000000000, 0xfffffa8005fe0570, 0x0000000000000002, 0xffffffffffffffff)
There were some other peculiar thing in the system log, like the firmware limitting certain processors, possibly system getting to hot.
Main cause of crash seems to be failure of system to respond in time, possibly caused by out of memory situation.
I checked the youtube stream nothing is visible on it concerning the crash lol. It would be diserably to have a seperate system for live streaming
to catch these crashes lol, not that it matters that much, just a funny thing to see/notice, but could also be used to attack windows systems !
--- SoupGate-Win32 v1.05
* Origin: fsxNet Usenet Gateway (21:1/5)