Shannon did some experiments to determine the entropy in English texts.
A later
work done by Cover and King [1] gave an estimate of 1.34 bits per
letter. This
implies that, if the letters are coded into 5 bits, one needs to appropriately
combine 4 text files in order to obtain bit sequences of full entropy, since 4*1.34 = 5.36 > 5. The method used in our software is to sum (mod 32)
the coded
values of a-z (mapped to 0-25) as 5 bits of the corresponding letters of
the
text files.
There are plenty of other schemes for obtaining high quality pseudo-random sequences in practice, e.g. AES in counter mode. However our scheme seems to be much simpler both in the underlying logic (understandability) and in implementation and is thus a viable alternative that one could use/need
under
circumstances.
The software, TEXTCOMBINE-SP, is available at http://mok-kong-shen.de
M. K. Shen -------------------------------------------------------------------------------
[1] T. M. Cover, R. C. King, A Convergent Gambling Estimate of the
Entropy of
English, IEEE Trans. Inf. Theory, vol. 24, 1978, pp. 413-421.
You are repeating yourself. Do you think that if you say it three times
(as with the Bellman) it will suddenly become worthwhile?
As I have said, this is a horrible scheme. text has many long range correlations (from charater pairs to paragraphs, etc) , which would mess up the random stream. (make it non-random).
Bad idea.
And you might want to look at what Shannon and others actually did.
Note that even if the letters really were completely random, your method
of combining them would make the output non-random.
On 2017-07-11, Mok-Kong Shen <mok-kong.shen@t-online.de> wrote:
Shannon did some experiments to determine the entropy in English texts.
A later
work done by Cover and King [1] gave an estimate of 1.34 bits per
letter. This
implies that, if the letters are coded into 5 bits, one needs to
appropriately
combine 4 text files in order to obtain bit sequences of full entropy, since >> 4*1.34 = 5.36 > 5. The method used in our software is to sum (mod 32)
the coded
values of a-z (mapped to 0-25) as 5 bits of the corresponding letters of
the
text files.
There are plenty of other schemes for obtaining high quality pseudo-random >> sequences in practice, e.g. AES in counter mode. However our scheme seems to >> be much simpler both in the underlying logic (understandability) and in
implementation and is thus a viable alternative that one could use/need
under
circumstances.
The software, TEXTCOMBINE-SP, is available at http://mok-kong-shen.de
M. K. Shen
-------------------------------------------------------------------------------
[1] T. M. Cover, R. C. King, A Convergent Gambling Estimate of the
Entropy of
English, IEEE Trans. Inf. Theory, vol. 24, 1978, pp. 413-421.
Am 11.07.2017 um 22:01 schrieb William Unruh:
You are repeating yourself. Do you think that if you say it three times
(as with the Bellman) it will suddenly become worthwhile?
As I have said, this is a horrible scheme. text has many long range
correlations (from charater pairs to paragraphs, etc) , which would
mess up the random stream. (make it non-random).
Bad idea.
And you might want to look at what Shannon and others actually did.
Note that even if the letters really were completely random, your method
of combining them would make the output non-random.
See the reference I gave of the paper about entropy and the test
statistic of Maurer's test.
(I had answered your post in another group and wonder why you didn't
answer there and switched to this one. I post to diverse groups because
the readers of different groups are not the same. Understand?)
M. K. Shen
On 2017-07-11, Mok-Kong Shen <mok-kong.shen@t-online.de> wrote:
Shannon did some experiments to determine the entropy in English texts.
A later
work done by Cover and King [1] gave an estimate of 1.34 bits per
letter. This
implies that, if the letters are coded into 5 bits, one needs to
appropriately
combine 4 text files in order to obtain bit sequences of full
entropy, since
4*1.34 = 5.36 > 5. The method used in our software is to sum (mod 32)
the coded
values of a-z (mapped to 0-25) as 5 bits of the corresponding letters of >>> the
text files.
There are plenty of other schemes for obtaining high quality
pseudo-random
sequences in practice, e.g. AES in counter mode. However our scheme
seems to
be much simpler both in the underlying logic (understandability) and in
implementation and is thus a viable alternative that one could use/need
under
circumstances.
The software, TEXTCOMBINE-SP, is available at http://mok-kong-shen.de
M. K. Shen
-------------------------------------------------------------------------------
[1] T. M. Cover, R. C. King, A Convergent Gambling Estimate of the
Entropy of
English, IEEE Trans. Inf. Theory, vol. 24, 1978, pp. 413-421.
On 12.07.17 17:37, Mok-Kong Shen wrote:
Am 11.07.2017 um 22:01 schrieb William Unruh:Did it ever occur to you that verifying randomness only by the Maurer
You are repeating yourself. Do you think that if you say it three times
(as with the Bellman) it will suddenly become worthwhile?
As I have said, this is a horrible scheme. text has many long range
correlations (from charater pairs to paragraphs, etc) , which would
mess up the random stream. (make it non-random).
Bad idea.
And you might want to look at what Shannon and others actually did.
Note that even if the letters really were completely random, your method >>> of combining them would make the output non-random.
See the reference I gave of the paper about entropy and the test
statistic of Maurer's test.
test is not sufficient?
What about ENT, test for bias or, much more important, Pierre L'Ecuyer's TestU01 suite?
In the past I have already demonstrated that one of your PRNG's, namely PERMPOLYPRNG which passed the Maurer test, is massively flawed.
Am 12.07.2017 um 18:44 schrieb Karl.Frank:
On 12.07.17 17:37, Mok-Kong Shen wrote:
Am 11.07.2017 um 22:01 schrieb William Unruh:Did it ever occur to you that verifying randomness only by the Maurer
You are repeating yourself. Do you think that if you say it three times >>>> (as with the Bellman) it will suddenly become worthwhile?
As I have said, this is a horrible scheme. text has many long range
correlations (from charater pairs to paragraphs, etc) , which would
mess up the random stream. (make it non-random).
Bad idea.
And you might want to look at what Shannon and others actually did.
Note that even if the letters really were completely random, your
method
of combining them would make the output non-random.
See the reference I gave of the paper about entropy and the test
statistic of Maurer's test.
test is not sufficient?
What about ENT, test for bias or, much more important, Pierre L'Ecuyer's
TestU01 suite?
In the past I have already demonstrated that one of your PRNG's, namely
PERMPOLYPRNG which passed the Maurer test, is massively flawed.
The unfortunate situation with PRN generation in general is that there
are lots of different tests. I don't have expertise in such and use just
one test for the sake of convenience. Further, my targeted users, the
common people who need security protection of their personal
communications, have only very limited volumes, so cryptanalytical risks associated with assumptions of large volumes of encrypted materials are
not intimidating for them IMHO.
M. K. Shen
On 12.07.17 22:44, Mok-Kong Shen wrote:
Am 12.07.2017 um 18:44 schrieb Karl.Frank:
On 12.07.17 17:37, Mok-Kong Shen wrote:
Am 11.07.2017 um 22:01 schrieb William Unruh:Did it ever occur to you that verifying randomness only by the Maurer
You are repeating yourself. Do you think that if you say it three
times
(as with the Bellman) it will suddenly become worthwhile?
As I have said, this is a horrible scheme. text has many long range
correlations (from charater pairs to paragraphs, etc) , which would
mess up the random stream. (make it non-random).
Bad idea.
And you might want to look at what Shannon and others actually did.
Note that even if the letters really were completely random, your
method
of combining them would make the output non-random.
See the reference I gave of the paper about entropy and the test
statistic of Maurer's test.
test is not sufficient?
What about ENT, test for bias or, much more important, Pierre L'Ecuyer's >>> TestU01 suite?
In the past I have already demonstrated that one of your PRNG's, namely
PERMPOLYPRNG which passed the Maurer test, is massively flawed.
The unfortunate situation with PRN generation in general is that there
are lots of different tests. I don't have expertise in such and use just
one test for the sake of convenience. Further, my targeted users, the
common people who need security protection of their personal
communications, have only very limited volumes, so cryptanalytical risks
associated with assumptions of large volumes of encrypted materials are
not intimidating for them IMHO.
M. K. Shen
It is not an unfortunate situation. In contrast these test suites are of great help in revealing weak or even totally useless PRNG's for the
common people - let alone in terms of cryptography.
So now you're promoting another cryptographic scheme for the obfuscation
of your little sisters diary? - as Bruce Schneier called it once.
Am 13.07.2017 um 02:15 schrieb Karl.Frank:
On 12.07.17 22:44, Mok-Kong Shen wrote:
Am 12.07.2017 um 18:44 schrieb Karl.Frank:
On 12.07.17 17:37, Mok-Kong Shen wrote:
Am 11.07.2017 um 22:01 schrieb William Unruh:Did it ever occur to you that verifying randomness only by the Maurer
You are repeating yourself. Do you think that if you say it three
times
(as with the Bellman) it will suddenly become worthwhile?
As I have said, this is a horrible scheme. text has many long range >>>>>> correlations (from charater pairs to paragraphs, etc) , which would >>>>>> mess up the random stream. (make it non-random).
Bad idea.
And you might want to look at what Shannon and others actually did. >>>>>>
Note that even if the letters really were completely random, your
method
of combining them would make the output non-random.
See the reference I gave of the paper about entropy and the test
statistic of Maurer's test.
test is not sufficient?
What about ENT, test for bias or, much more important, Pierre
L'Ecuyer's
TestU01 suite?
In the past I have already demonstrated that one of your PRNG's, namely >>>> PERMPOLYPRNG which passed the Maurer test, is massively flawed.
The unfortunate situation with PRN generation in general is that there
are lots of different tests. I don't have expertise in such and use just >>> one test for the sake of convenience. Further, my targeted users, the
common people who need security protection of their personal
communications, have only very limited volumes, so cryptanalytical risks >>> associated with assumptions of large volumes of encrypted materials are
not intimidating for them IMHO.
M. K. Shen
It is not an unfortunate situation. In contrast these test suites are of
great help in revealing weak or even totally useless PRNG's for the
common people - let alone in terms of cryptography.
So now you're promoting another cryptographic scheme for the obfuscation
of your little sisters diary? - as Bruce Schneier called it once.
If the volume of materials available for analyze is small, then
exploiting tiny biases would be more difficult, isn't it?
Views could indeed be entirely different. Schneier demanded also that
one who is interested in crypto should first be proficient in analyzing
the diverse classical schemes. If everyone follows that advice, I am
quite sure that a non-trivial percentage of persons currently in the
crypto groups would have been absent because they haven't yet been able
to finish the work that they are required to do.
M. K. Shen
Am 13.07.2017 um 11:39 schrieb Karl.Frank:
On 13.07.17 08:36, Mok-Kong Shen wrote:
Am 13.07.2017 um 02:15 schrieb Karl.Frank:Depending on how heavy the bias of the PRNG in question is. You might
On 12.07.17 22:44, Mok-Kong Shen wrote:
Am 12.07.2017 um 18:44 schrieb Karl.Frank:
On 12.07.17 17:37, Mok-Kong Shen wrote:
Am 11.07.2017 um 22:01 schrieb William Unruh:Did it ever occur to you that verifying randomness only by the Maurer >>>>>> test is not sufficient?
You are repeating yourself. Do you think that if you say it three >>>>>>>> times
(as with the Bellman) it will suddenly become worthwhile?
As I have said, this is a horrible scheme. text has many long range >>>>>>>> correlations (from charater pairs to paragraphs, etc) , which would >>>>>>>> mess up the random stream. (make it non-random).
Bad idea.
And you might want to look at what Shannon and others actually did. >>>>>>>>
Note that even if the letters really were completely random, your >>>>>>>> method
of combining them would make the output non-random.
See the reference I gave of the paper about entropy and the test >>>>>>> statistic of Maurer's test.
What about ENT, test for bias or, much more important, Pierre
L'Ecuyer's
TestU01 suite?
In the past I have already demonstrated that one of your PRNG's,
namely
PERMPOLYPRNG which passed the Maurer test, is massively flawed.
The unfortunate situation with PRN generation in general is that there >>>>> are lots of different tests. I don't have expertise in such and use
just
one test for the sake of convenience. Further, my targeted users, the >>>>> common people who need security protection of their personal
communications, have only very limited volumes, so cryptanalytical
risks
associated with assumptions of large volumes of encrypted materials
are
not intimidating for them IMHO.
M. K. Shen
It is not an unfortunate situation. In contrast these test suites
are of
great help in revealing weak or even totally useless PRNG's for the
common people - let alone in terms of cryptography.
So now you're promoting another cryptographic scheme for the
obfuscation
of your little sisters diary? - as Bruce Schneier called it once.
If the volume of materials available for analyze is small, then
exploiting tiny biases would be more difficult, isn't it?
notice that the most recent break of RC4 was basically managed with very
tiny ciphertexts.
I don't yet know that recent break. How tiny? Could you give a
reference?
BTW, you wrote earlier:
"In the past I have already demonstrated that one of your PRNG's,
namely PERMPOLYPRNG which passed the Maurer test, is massively flawed."
The latest version of that software is 3.1, so at least its first
version was fairly unsatisfactory even to myself. However, all the
revisions were based on thoughts of myself, not of any other person. It
can certainly not be excluded that even the latest version may indeed
be "massively flawed". But where is your "demonstration"?? I just
checked and found that the thread in the group where PERMPOLYPRNG is published is exceptionally short and all posts in it were from me and
not from any other person.
M. K. Shen
Views could indeed be entirely different. Schneier demanded also thatThis would hold mostly true for you then.
one who is interested in crypto should first be proficient in analyzing
the diverse classical schemes. If everyone follows that advice, I am
quite sure that a non-trivial percentage of persons currently in the
crypto groups would have been absent because they haven't yet been able
to finish the work that they are required to do.
M. K. Shen
On 13.07.17 08:36, Mok-Kong Shen wrote:
Am 13.07.2017 um 02:15 schrieb Karl.Frank:Depending on how heavy the bias of the PRNG in question is. You might
On 12.07.17 22:44, Mok-Kong Shen wrote:
Am 12.07.2017 um 18:44 schrieb Karl.Frank:
On 12.07.17 17:37, Mok-Kong Shen wrote:
Am 11.07.2017 um 22:01 schrieb William Unruh:Did it ever occur to you that verifying randomness only by the Maurer >>>>> test is not sufficient?
You are repeating yourself. Do you think that if you say it three >>>>>>> times
(as with the Bellman) it will suddenly become worthwhile?
As I have said, this is a horrible scheme. text has many long range >>>>>>> correlations (from charater pairs to paragraphs, etc) , which would >>>>>>> mess up the random stream. (make it non-random).
Bad idea.
And you might want to look at what Shannon and others actually did. >>>>>>>
Note that even if the letters really were completely random, your >>>>>>> method
of combining them would make the output non-random.
See the reference I gave of the paper about entropy and the test
statistic of Maurer's test.
What about ENT, test for bias or, much more important, Pierre
L'Ecuyer's
TestU01 suite?
In the past I have already demonstrated that one of your PRNG's,
namely
PERMPOLYPRNG which passed the Maurer test, is massively flawed.
The unfortunate situation with PRN generation in general is that there >>>> are lots of different tests. I don't have expertise in such and use
just
one test for the sake of convenience. Further, my targeted users, the
common people who need security protection of their personal
communications, have only very limited volumes, so cryptanalytical
risks
associated with assumptions of large volumes of encrypted materials are >>>> not intimidating for them IMHO.
M. K. Shen
It is not an unfortunate situation. In contrast these test suites are of >>> great help in revealing weak or even totally useless PRNG's for the
common people - let alone in terms of cryptography.
So now you're promoting another cryptographic scheme for the obfuscation >>> of your little sisters diary? - as Bruce Schneier called it once.
If the volume of materials available for analyze is small, then
exploiting tiny biases would be more difficult, isn't it?
notice that the most recent break of RC4 was basically managed with very
tiny ciphertexts.
Views could indeed be entirely different. Schneier demanded also thatThis would hold mostly true for you then.
one who is interested in crypto should first be proficient in analyzing
the diverse classical schemes. If everyone follows that advice, I am
quite sure that a non-trivial percentage of persons currently in the
crypto groups would have been absent because they haven't yet been able
to finish the work that they are required to do.
M. K. Shen
On 13.07.17 13:29, Mok-Kong Shen wrote:
Am 13.07.2017 um 11:39 schrieb Karl.Frank:Just tiny short 16-character cookies.
On 13.07.17 08:36, Mok-Kong Shen wrote:
Am 13.07.2017 um 02:15 schrieb Karl.Frank:Depending on how heavy the bias of the PRNG in question is. You might
On 12.07.17 22:44, Mok-Kong Shen wrote:
Am 12.07.2017 um 18:44 schrieb Karl.Frank:
On 12.07.17 17:37, Mok-Kong Shen wrote:
Am 11.07.2017 um 22:01 schrieb William Unruh:Did it ever occur to you that verifying randomness only by the
You are repeating yourself. Do you think that if you say it three >>>>>>>>> times
(as with the Bellman) it will suddenly become worthwhile?
As I have said, this is a horrible scheme. text has many long >>>>>>>>> range
correlations (from charater pairs to paragraphs, etc) , which >>>>>>>>> would
mess up the random stream. (make it non-random).
Bad idea.
And you might want to look at what Shannon and others actually >>>>>>>>> did.
Note that even if the letters really were completely random, your >>>>>>>>> method
of combining them would make the output non-random.
See the reference I gave of the paper about entropy and the test >>>>>>>> statistic of Maurer's test.
Maurer
test is not sufficient?
What about ENT, test for bias or, much more important, Pierre
L'Ecuyer's
TestU01 suite?
In the past I have already demonstrated that one of your PRNG's, >>>>>>> namely
PERMPOLYPRNG which passed the Maurer test, is massively flawed.
The unfortunate situation with PRN generation in general is that
there
are lots of different tests. I don't have expertise in such and use >>>>>> just
one test for the sake of convenience. Further, my targeted users, the >>>>>> common people who need security protection of their personal
communications, have only very limited volumes, so cryptanalytical >>>>>> risks
associated with assumptions of large volumes of encrypted materials >>>>>> are
not intimidating for them IMHO.
M. K. Shen
It is not an unfortunate situation. In contrast these test suites
are of
great help in revealing weak or even totally useless PRNG's for the
common people - let alone in terms of cryptography.
So now you're promoting another cryptographic scheme for the
obfuscation
of your little sisters diary? - as Bruce Schneier called it once.
If the volume of materials available for analyze is small, then
exploiting tiny biases would be more difficult, isn't it?
notice that the most recent break of RC4 was basically managed with very >>> tiny ciphertexts.
I don't yet know that recent break. How tiny? Could you give a
reference?
http://www.rc4nomore.com/
http://www.youtube.com/watch?v=d8MtmKrXlKQ
http://www.rc4nomore.com/vanhoef-usenix2015.pdf
BTW, you wrote earlier:Over here are some visual results displaying the massive bias as well as
"In the past I have already demonstrated that one of your PRNG's,
namely PERMPOLYPRNG which passed the Maurer test, is massively flawed."
The latest version of that software is 3.1, so at least its first
version was fairly unsatisfactory even to myself. However, all the
revisions were based on thoughts of myself, not of any other person. It
can certainly not be excluded that even the latest version may indeed
be "massively flawed". But where is your "demonstration"?? I just
checked and found that the thread in the group where PERMPOLYPRNG is
published is exceptionally short and all posts in it were from me and
not from any other person.
the source code and test tools used http://www.freecx.co.uk/crypto/cryptanalysis/PERMPOLYPRNG/
Especially these two images are mostly interesting in regards of
displaying the bias
http://www.freecx.co.uk/crypto/cryptanalysis/PERMPOLYPRNG/permpolyprng_1MB.bin_rnd.jpg
http://www.freecx.co.uk/crypto/cryptanalysis/PERMPOLYPRNG/permpolyprng_4MB.bin_rnd.jpg
The original posting is over here http://s13.zetaboards.com/Crypto/single/?p=8045140&t=7392575
M. K. Shen
Views could indeed be entirely different. Schneier demanded also thatThis would hold mostly true for you then.
one who is interested in crypto should first be proficient in analyzing >>>> the diverse classical schemes. If everyone follows that advice, I am
quite sure that a non-trivial percentage of persons currently in the
crypto groups would have been absent because they haven't yet been able >>>> to finish the work that they are required to do.
M. K. Shen
Am 13.07.2017 um 14:08 schrieb Karl.Frank:
On 13.07.17 13:29, Mok-Kong Shen wrote:
Am 13.07.2017 um 11:39 schrieb Karl.Frank:Just tiny short 16-character cookies.
On 13.07.17 08:36, Mok-Kong Shen wrote:
Am 13.07.2017 um 02:15 schrieb Karl.Frank:Depending on how heavy the bias of the PRNG in question is. You might
On 12.07.17 22:44, Mok-Kong Shen wrote:
Am 12.07.2017 um 18:44 schrieb Karl.Frank:
On 12.07.17 17:37, Mok-Kong Shen wrote:The unfortunate situation with PRN generation in general is that >>>>>>> there
Am 11.07.2017 um 22:01 schrieb William Unruh:Did it ever occur to you that verifying randomness only by the >>>>>>>> Maurer
You are repeating yourself. Do you think that if you say it three >>>>>>>>>> times
(as with the Bellman) it will suddenly become worthwhile?
As I have said, this is a horrible scheme. text has many long >>>>>>>>>> range
correlations (from charater pairs to paragraphs, etc) , which >>>>>>>>>> would
mess up the random stream. (make it non-random).
Bad idea.
And you might want to look at what Shannon and others actually >>>>>>>>>> did.
Note that even if the letters really were completely random, your >>>>>>>>>> method
of combining them would make the output non-random.
See the reference I gave of the paper about entropy and the test >>>>>>>>> statistic of Maurer's test.
test is not sufficient?
What about ENT, test for bias or, much more important, Pierre
L'Ecuyer's
TestU01 suite?
In the past I have already demonstrated that one of your PRNG's, >>>>>>>> namely
PERMPOLYPRNG which passed the Maurer test, is massively flawed. >>>>>>>
are lots of different tests. I don't have expertise in such and use >>>>>>> just
one test for the sake of convenience. Further, my targeted users, >>>>>>> the
common people who need security protection of their personal
communications, have only very limited volumes, so cryptanalytical >>>>>>> risks
associated with assumptions of large volumes of encrypted materials >>>>>>> are
not intimidating for them IMHO.
M. K. Shen
It is not an unfortunate situation. In contrast these test suites
are of
great help in revealing weak or even totally useless PRNG's for the >>>>>> common people - let alone in terms of cryptography.
So now you're promoting another cryptographic scheme for the
obfuscation
of your little sisters diary? - as Bruce Schneier called it once.
If the volume of materials available for analyze is small, then
exploiting tiny biases would be more difficult, isn't it?
notice that the most recent break of RC4 was basically managed with
very
tiny ciphertexts.
I don't yet know that recent break. How tiny? Could you give a
reference?
http://www.rc4nomore.com/
http://www.youtube.com/watch?v=d8MtmKrXlKQ
http://www.rc4nomore.com/vanhoef-usenix2015.pdf
BTW, you wrote earlier:Over here are some visual results displaying the massive bias as well as
"In the past I have already demonstrated that one of your PRNG's,
namely PERMPOLYPRNG which passed the Maurer test, is massively flawed."
The latest version of that software is 3.1, so at least its first
version was fairly unsatisfactory even to myself. However, all the
revisions were based on thoughts of myself, not of any other person. It
can certainly not be excluded that even the latest version may indeed
be "massively flawed". But where is your "demonstration"?? I just
checked and found that the thread in the group where PERMPOLYPRNG is
published is exceptionally short and all posts in it were from me and
not from any other person.
the source code and test tools used
http://www.freecx.co.uk/crypto/cryptanalysis/PERMPOLYPRNG/
Especially these two images are mostly interesting in regards of
displaying the bias
http://www.freecx.co.uk/crypto/cryptanalysis/PERMPOLYPRNG/permpolyprng_1MB.bin_rnd.jpg
http://www.freecx.co.uk/crypto/cryptanalysis/PERMPOLYPRNG/permpolyprng_4MB.bin_rnd.jpg
The original posting is over here
http://s13.zetaboards.com/Crypto/single/?p=8045140&t=7392575
Ok, I forgot that history, as it laid quite a time back. From the date
of that post of yours, you were testing version 1.0 not the later
versions.
I dont have the opinion that any crypto scheme be justified only by its passing a single statistical test. When some other plausible reasoning
are available in the positive direction, then such a test gives in my
view substantial support for its goodness. Formal proof of security
would be ideal, but in practice that's a difficult to attain goal.
I can't remember/know now how much work I had spent to check version
1.0 with Maurer's test and whether I might have done mistakes there and
so any bad behavior of Version 1.0 should not be interpreted to be non-sensitiveness of Maurer's test. In fact, in the current case of TESTCOMBINE-SP, certain arguments seemed to indicate that its resulting sequences would be fairly biased though Maurer's test came out always to
be ok and I started to doubt the sensitivity of Maurer's test. But, if
my later computations are correct, this can be explained by the fact
that the underlying factors of the said arguments turned out not to
be strong enough in their influence in practice and hence Maurer's test
can't be blamed in that context.
M. K. Shen
M. K. Shen
Views could indeed be entirely different. Schneier demanded also that >>>>> one who is interested in crypto should first be proficient inThis would hold mostly true for you then.
analyzing
the diverse classical schemes. If everyone follows that advice, I am >>>>> quite sure that a non-trivial percentage of persons currently in the >>>>> crypto groups would have been absent because they haven't yet been
able
to finish the work that they are required to do.
M. K. Shen
Concerning the reference you gave of the break of RC-4, I read there
"Our attack is not limited to decrypting cookies. Any data or
information that is repeatedly encrypted can be recovered".
Do you know how much is this the special fault of RC-4? I mean, would
other PRNGs also be liable to that attack just as effectively?
M. K. Shen
On 2017-07-13, Mok-Kong Shen <mok-kong.shen@t-online.de> wrote:
Concerning the reference you gave of the break of RC-4, I read there
"Our attack is not limited to decrypting cookies. Any data or
information that is repeatedly encrypted can be recovered".
Do you know how much is this the special fault of RC-4? I mean, would
other PRNGs also be liable to that attack just as effectively?
It is in that case a fault of RC4. It had long been known that RC4 had biases, especially in the first bytes that came out of RC4. These
attacks show that those biases are
useable in an attack. RC4 is broken.
Note that your scheme here seems to be using 4 text files as a "key" Ie,
to encrypt 1000 plaintexts you need to find 4000 different text fiels to
use, and you have to communicate to the recipient what those files are without also telling the attacker what those files are. Plus as I said,
your ouput will have loads of biases in it simply because of hthe strong
long range correlations in any text file.
Any PRNG used for cryptographic purpose *HAS* to *pass* *all* of these
very harsh and intense tests for randomness quality, especially TestU01 "crush" and "big crush". This is the first ever measurement that a PRNG designer has to take very seriously. If a proposed CSPRNG does not pass
these test it has to be dropped or re-designed, because a *failure* of
these test *indicate* a *non-random* *output* no matter what one
believes are the plausible reasons why it would still be "sufficiently random" for cryptographic purposes.
If you would read the mentioned thread from the beginning you will
realised that my intention on starting it was my critique that the
Maurer test is seemingly not reliable, as even the keystream output of
one of the most miserably designed cipher algorithms, namely the
Crystalline cipher, passes the Maurer test. This indicates in my view
the importance not to rely solely on this result but always run the
whole bunch of available test tools.
Just one example of the Crystalline output by the Maurer test http://www.freecx.co.uk/crypto/cryptanalysis/Crystalline/maurer-test-result_10MB.txt
...and that's what the simple test for bias reveals http://www.freecx.co.uk/crypto/cryptanalysis/Crystalline/bias-result_(8)_10MB.txt
Both test results are based on the same keystream.
Not sure if the latest version of your PERMPOLYPRNG or TESTCOMBINE-SP
would pass these test.
Am 13.07.2017 um 18:02 schrieb Karl.Frank:
Any PRNG used for cryptographic purpose *HAS* to *pass* *all* of these
very harsh and intense tests for randomness quality, especially TestU01
"crush" and "big crush". This is the first ever measurement that a PRNG
designer has to take very seriously. If a proposed CSPRNG does not pass
these test it has to be dropped or re-designed, because a *failure* of
these test *indicate* a *non-random* *output* no matter what one
believes are the plausible reasons why it would still be "sufficiently
random" for cryptographic purposes.
If you would read the mentioned thread from the beginning you will
realised that my intention on starting it was my critique that the
Maurer test is seemingly not reliable, as even the keystream output of
one of the most miserably designed cipher algorithms, namely the
Crystalline cipher, passes the Maurer test. This indicates in my view
the importance not to rely solely on this result but always run the
whole bunch of available test tools.
Just one example of the Crystalline output by the Maurer test
http://www.freecx.co.uk/crypto/cryptanalysis/Crystalline/maurer-test-result_10MB.txt
...and that's what the simple test for bias reveals
http://www.freecx.co.uk/crypto/cryptanalysis/Crystalline/bias-result_(8)_10MB.txt
Both test results are based on the same keystream.
Not sure if the latest version of your PERMPOLYPRNG or TESTCOMBINE-SP
would pass these test.
I know naturally that, if one employs more different kinds of tests,
that's anyway better than less and that this is in fact true for all
field of science. Long time ago I thought of the NIST test suite.
However, my OS is Windows and I read on Internet that there were some difficulties to have that test suite run on Windows and dropped the
idea. Which good alternatives in your experience run on Windows straightforwardly?
M. K. Shen
Am 13.07.2017 um 17:40 schrieb William Unruh:
On 2017-07-13, Mok-Kong Shen <mok-kong.shen@t-online.de> wrote:
Concerning the reference you gave of the break of RC-4, I read there
"Our attack is not limited to decrypting cookies. Any data or
information that is repeatedly encrypted can be recovered".
Do you know how much is this the special fault of RC-4? I mean, would
other PRNGs also be liable to that attack just as effectively?
It is in that case a fault of RC4. It had long been known that RC4 had
biases, especially in the first bytes that came out of RC4. These
attacks show that those biases are
useable in an attack. RC4 is broken.
Note that your scheme here seems to be using 4 text files as a "key" Ie,
to encrypt 1000 plaintexts you need to find 4000 different text fiels to
use, and you have to communicate to the recipient what those files are
without also telling the attacker what those files are. Plus as I said,
your ouput will have loads of biases in it simply because of hthe strong
long range correlations in any text file.
In fact I had also done a simple counting to see whether the least significant bit of my particular encoding of input letters leads to
strong bias, for it seems that that bit could be sensitive. It turned
out that the ratio of counts of 0 to total counts varies for the
individual files are in the range of [0.52, 0.57}. When 2 files are
combined with xor this range is reduced. With 4 files the range is
reduced to almost exactly 0.5.
M. K. Shen
Just counting the appearance of zeros and ones is not sufficient.[snip]
Important is how they are spread over the whole output. As an example
you might be interested in this particular explanation found on WikiPedia
On 2017-07-13, Mok-Kong Shen<mok-kong.shen@t-online.de> wrote:
Concerning the reference you gave of the break of RC-4, I read there
"Our attack is not limited to decrypting cookies. Any data or
information that is repeatedly encrypted can be recovered".
Do you know how much is this the special fault of RC-4? I mean, would
other PRNGs also be liable to that attack just as effectively?
It is in that case a fault of RC4. It had long been known that RC4 had biases, especially in the first bytes that came out of RC4. These
attacks show that those biases are
useable in an attack. RC4 is broken.
Note that your scheme here seems to be using 4 text files as a "key" Ie,
to encrypt 1000 plaintexts you need to find 4000 different text fiels to
use, and you have to communicate to the recipient what those files are without also telling the attacker what those files are. Plus as I said,
your ouput will have loads of biases in it simply because of hthe strong
long range correlations in any text file.
M. K. Shen
Am 13.07.2017 um 21:08 schrieb Karl.Frank:
Just counting the appearance of zeros and ones is not sufficient.[snip]
Important is how they are spread over the whole output. As an example
you might be interested in this particular explanation found on WikiPedia
The correlation is to my knowledge commonly investigated with the autocorrelation test. Maurer's test should in a sense be a superset
covering that test. To check that I'll code the autocorrelation test
and apply it to the result of my example and hope to be able to report
on that issue not later than tomorrow evening anyway.
M. K. Shen
On 13.07.17 17:40, William Unruh wrote:
On 2017-07-13, Mok-Kong Shen<mok-kong.shen@t-online.de> wrote:This sounds like a generation of some sort of one-time-pad to me,
Concerning the reference you gave of the break of RC-4, I read there
"Our attack is not limited to decrypting cookies. Any data or
information that is repeatedly encrypted can be recovered".
Do you know how much is this the special fault of RC-4? I mean, would
other PRNGs also be liable to that attack just as effectively?
It is in that case a fault of RC4. It had long been known that RC4 had
biases, especially in the first bytes that came out of RC4. These
attacks show that those biases are
useable in an attack. RC4 is broken.
Note that your scheme here seems to be using 4 text files as a "key" Ie,
to encrypt 1000 plaintexts you need to find 4000 different text fiels to
use, and you have to communicate to the recipient what those files are
without also telling the attacker what those files are. Plus as I said,
your ouput will have loads of biases in it simply because of hthe strong
long range correlations in any text file.
inheriting the key exchange problem as well as the necessity to keep
track of which files already used etc..., rendering it extremely useless
to the "common people" which it was originally designed for.
But regarding the bias I consider that it /might/ be possible to
generate a good pseudo-random encryption key if we would XOR four JPEG
files of nearly the same size and of which the header and footer is
dropped.
Or even better, if we seed a regular PRNG and now cycle over these files byte-wise, picking four byte based on four consecutive 8bit values drawn
from the PRNG on each step. Once we reached the end of the files we
start again at the top. Of course this implies a PRNG that has a very
long period and can be seeded with at least a 64bit seed. This way we
might share the four basic key files as innocent JPEG images.
M. K. Shen
| Sysop: | Keyop |
|---|---|
| Location: | Huddersfield, West Yorkshire, UK |
| Users: | 296 |
| Nodes: | 16 (2 / 14) |
| Uptime: | 79:55:47 |
| Calls: | 6,658 |
| Calls today: | 4 |
| Files: | 12,203 |
| Messages: | 5,333,178 |
| Posted today: | 1 |
