Hello,
About supercharging your bash workflows with GNU parallel..
I invite you to read the following article:
How to supercharge your bash workflows with GNU parallel
https://www.freecodecamp.org/news/how-to-supercharge-your-bash-workflows-with-gnu-parallel-53aab0aea141/
I think the GNU parallel is not the right tool, because
you have to abstract it much more correctly so that to avoid
thread and process switching that hurts scalability, so then in case
of GNU parallel you are required to take a look at the bash scripts
source code and understand them so that to avoid excessive thread and
process switching that hurts scalability, so i think that GNU parallel
is not the right tool, so you have to use parallel "processing" between
bash scripts so that to easy the job of managing scalability.
So i invite you to read my following thoughts about my new software inventions..
And my today software invention is the following:
You have to know that a Turing-complete system can be proven
mathematically to be capable of performing any possible calculation or
computer program, and bash shell for Linux and Windows are
Turing-complete, and even if bash shell is not python, it is a
minimalist language that is especially designed for administrators of
operating systems, but i have noticed that bash shell is not suited for
for parallel programming, this is why i am enhancing it with my
scalable algorithms so that to support sophisticated parallel
programming on both Linux and Windows that permits it to scale much
better on RAIDs and on multicores. So i am also writing a book about my enhancement to bash shell with my scalable algorithms so that to help
others be efficient in bash shell programming and efficient in operating
system administration, and of course i will sell my book, so i don't
think you need python since python doesn't come with my scalable
algorithms that will enhance bash for Linux and Windows, and i think
operating systems administrators don't need python since it is
not suited for operating system administrators since it is
not a minimalist language as bash for Linux and Windows.
You can read more about bash shell from here:
https://www.infoworld.com/article/2893519/perl-python-ruby-are-nice-bash-is-where-its-at.html
Also my next software invention is the following:
More philosophy about what is artificial intelligence and more..
I have also invented many scalable algorithms and algorithms and i am
finding artificial intelligence easy to learn, i think to be able to
understand artificial intelligence you have to understand reasoning with
energy minimization, like with PSO(Particle Swarm Optimization), but
you have to be smart since the Population based algorithm has to
guarantee the optimal convergence, and this is why i am learning
you how to do it(read below), i think that GA(genetic algorithm) is
good for teaching it, but GA(genetic algorithm) doesn't guarantee the
optimal convergence, and after learning how to do reasoning with energy minimization in artificial intelligence, you have to understand what is transfer learning in artificial intelligence with PathNet or such, this transfer learning permits to train faster and require less labeled data,
also PathNET is much more powerful since also it is higher level
abstraction in artificial intelligence..
Read about it here:
https://mattturck.com/frontierai/
And read about PathNet here:
https://medium.com/@thoszymkowiak/deepmind-just-published-a-mind-blowing-paper-pathnet-f72b1ed38d46
More about artificial intelligence..
I think one of the most important part in artificial intelligence is
reasoning with energy minimization, it is the one that i am working on
right now, see the following video to understand more about it:
Yann LeCun: Can Neural Networks Reason?
https://www.youtube.com/watch?v=YAfwNEY826I&t=250s
I think that since i have just understood much more artificial
intelligence, i will soon show you my next Open source software project
that implement a powerful much more scalable Parallel Linear programming
solver and a powerful much more scalable Parallel Mixed-integer
programming solver with Artificial intelligence using PSO, and i will
write an article that explain much more artificial intelligence and what
is smartness and what is consciousness and self-awareness..
And in only one day i have just learned "much" more artificial
intelligence, i have read the following article about Particle Swarm Optimization and i have understood it:
Artificial Intelligence - Particle Swarm Optimization
https://docs.microsoft.com/en-us/archive/msdn-magazine/2011/august/artificial-intelligence-particle-swarm-optimization
But i have just noticed that the above implementation doesn't guarantee
the optimal convergence.
So here is how to guarantee the optimal convergence in PSO:
Clerc and Kennedy in (Trelea 2003) propose a constriction coefficient
parameter selection guidelines in order to guarantee the optimal
convergence, here is how to do it with PSO:
v(t+1) = k*[(v(t) + (c1 * r1 * (p(t) – x(t)) + (c2 * r2 * (g(t) – x(t))]
x(t+1) = x(t) + v(t+1)
constriction coefficient parameter is:
k = 2/abs(2-phi-sqrt(phi^2-(4*phi)))
k:=2/abs((2-4.1)-(0.640)) = 0.729
phi = c1 + c2
To guarantee the optimal convergence use:
c1 = c2 = 2.05
phi = 4.1 => k equal to 0.729
w=0.7298
Population size = 60;
Also i have noticed that GA(genetic algorithm) doesn't guarantee the
optimal convergence, and SA(Simulated annealing) and Hill Climbing are
much less powerful since they perform only exploitation.
In general, any metaheuristic should perform two main searching
capabilities (Exploration and Exploitation). Population based algorithms
( or many solutions ) such as GA, PSO, ACO, or ABC, performs both
Exploration and Exploitation, while Single-Based Algorithm such as SA(Simulated annealing), Hill Climbing, performs the exploitation only.
In this case, more exploitation and less exploration increases the
chances for trapping in local optima. Because the algorithm does not
have the ability to search in another position far from the current best solution ( which is Exploration).
Simulated annealing starts in one valley and typically ends in the
lowest point of the same valley. Whereas swarms start in many different
places of the mountain range and are searching for the lowest point in
many valleys simultaneously.
And in my next Open source software project i will implement a powerful
much more scalable Parallel Linear programming solver and a powerful
much more scalable Parallel Mixed-integer programming solver with
Artificial intelligence using PSO.
Thank you,
Amine Moulay Ramdane.
--- SoupGate-Win32 v1.05
* Origin: fsxNet Usenet Gateway (21:1/5)