Hello all
I just wiped the dust off my old HP48GX in order to do some celestial navigation calculations. I like the function to convert HHMMSS into decimal hours, or degrees, if you will.
Everything works fine, I can convert back and forth. But this morning I found a result that was obviously fairly off.
When I calculate 11:54:54 minus 11:52:54 it should give me exactly 2 minutes, easy to see. But the operation on the HP results in 11.915 - 11.8816666667 = 0.333333333. This converted back to HHMMSS equals 0.015999999988, ie. 4 seconds off. Why?
Hello allWhat is the actual code that you are using to make the calculation?
I just wiped the dust off my old HP48GX in order to do some celestial navigation calculations. I like the function to convert HHMMSS into decimal hours, or degrees, if you will.
Everything works fine, I can convert back and forth. But this morning I found a result that was obviously fairly off.
When I calculate 11:54:54 minus 11:52:54 it should give me exactly 2 minutes, easy to see. But the operation on the HP results in 11.915 - 11.8816666667 = 0.333333333. This converted back to HHMMSS equals 0.015999999988, ie. 4 seconds off. Why?
Roundoff error. If you follow discussions of the cardinality of the reals or other such topics, you may recognize that 0.015999999999999999... is equivalent to .0160, i.e., 1 minute 60 seconds, i.e., 2 minutes. 0.015999999988 is a decent numericalapproximation to that repeating decimal.
On Wednesday, April 13, 2022 at 9:32:29 AM UTC-4, druid....@gmail.com wrote:Hello
Hello allWhat is the actual code that you are using to make the calculation?
I just wiped the dust off my old HP48GX in order to do some celestial navigation calculations. I like the function to convert HHMMSS into decimal hours, or degrees, if you will.
Everything works fine, I can convert back and forth. But this morning I found a result that was obviously fairly off.
When I calculate 11:54:54 minus 11:52:54 it should give me exactly 2 minutes, easy to see. But the operation on the HP results in 11.915 - 11.8816666667 = 0.333333333. This converted back to HHMMSS equals 0.015999999988, ie. 4 seconds off. Why?
optic...@mindspring.com schrieb am Donnerstag, 14. April 2022 um 03:57:30 UTC+2:approximation to that repeating decimal.
Roundoff error. If you follow discussions of the cardinality of the reals or other such topics, you may recognize that 0.015999999999999999... is equivalent to .0160, i.e., 1 minute 60 seconds, i.e., 2 minutes. 0.015999999988 is a decent numerical
below. Moreover, on a HP41 it gives the proper result of 2 minutes..On Wednesday, April 13, 2022 at 9:32:29 AM UTC-4, druid....@gmail.com wrote:Hello
Hello allWhat is the actual code that you are using to make the calculation?
I just wiped the dust off my old HP48GX in order to do some celestial navigation calculations. I like the function to convert HHMMSS into decimal hours, or degrees, if you will.
Everything works fine, I can convert back and forth. But this morning I found a result that was obviously fairly off.
When I calculate 11:54:54 minus 11:52:54 it should give me exactly 2 minutes, easy to see. But the operation on the HP results in 11.915 - 11.8816666667 = 0.333333333. This converted back to HHMMSS equals 0.015999999988, ie. 4 seconds off. Why?
I guess so. However, with all the calculations I have done so far, this has never occurred. When I convert HHMMSS into Decimals and add or subtract, then convert it back again into HHSSMM it is exact to the split second. But not in the specific example
Here is what I do: Enter, for example, 3:55:59 (3.5559 ENTER, convert to decimals with the built in function (Time, HMS->) add 1:59:20 (1.5920) by converting and convert the result back (->HMS) I get 5:11:19 which is correct. So, I don’t get thatrounding part, really.
optic...@mindspring.com schrieb am Donnerstag, 14. April 2022 um 03:57:30 UTC+2:approximation to that repeating decimal.
Roundoff error. If you follow discussions of the cardinality of the reals or other such topics, you may recognize that 0.015999999999999999... is equivalent to .0160, i.e., 1 minute 60 seconds, i.e., 2 minutes. 0.015999999988 is a decent numerical
below. Moreover, on a HP41 it gives the proper result of 2 minutes..On Wednesday, April 13, 2022 at 9:32:29 AM UTC-4, druid....@gmail.com wrote:Hello
Hello allWhat is the actual code that you are using to make the calculation?
I just wiped the dust off my old HP48GX in order to do some celestial navigation calculations. I like the function to convert HHMMSS into decimal hours, or degrees, if you will.
Everything works fine, I can convert back and forth. But this morning I found a result that was obviously fairly off.
When I calculate 11:54:54 minus 11:52:54 it should give me exactly 2 minutes, easy to see. But the operation on the HP results in 11.915 - 11.8816666667 = 0.333333333. This converted back to HHMMSS equals 0.015999999988, ie. 4 seconds off. Why?
I guess so. However, with all the calculations I have done so far, this has never occurred. When I convert HHMMSS into Decimals and add or subtract, then convert it back again into HHSSMM it is exact to the split second. But not in the specific example
Here is what I do: Enter, for example, 3:55:59 (3.5559 ENTER, convert to decimals with the built in function (Time, HMS->) add 1:59:20 (1.5920) by converting and convert the result back (->HMS) I get 5:11:19 which is correct. So, I don’t get thatrounding part, really.
On Friday, April 15, 2022 at 12:01:41 PM UTC-4, marc....@me.com wrote:approximation to that repeating decimal.
optic...@mindspring.com schrieb am Donnerstag, 14. April 2022 um 03:57:30 UTC+2:
Roundoff error. If you follow discussions of the cardinality of the reals or other such topics, you may recognize that 0.015999999999999999... is equivalent to .0160, i.e., 1 minute 60 seconds, i.e., 2 minutes. 0.015999999988 is a decent numerical
example below. Moreover, on a HP41 it gives the proper result of 2 minutes..On Wednesday, April 13, 2022 at 9:32:29 AM UTC-4, druid....@gmail.com wrote:Hello
Hello allWhat is the actual code that you are using to make the calculation?
I just wiped the dust off my old HP48GX in order to do some celestial navigation calculations. I like the function to convert HHMMSS into decimal hours, or degrees, if you will.
Everything works fine, I can convert back and forth. But this morning I found a result that was obviously fairly off.
When I calculate 11:54:54 minus 11:52:54 it should give me exactly 2 minutes, easy to see. But the operation on the HP results in 11.915 - 11.8816666667 = 0.333333333. This converted back to HHMMSS equals 0.015999999988, ie. 4 seconds off. Why?
I guess so. However, with all the calculations I have done so far, this has never occurred. When I convert HHMMSS into Decimals and add or subtract, then convert it back again into HHSSMM it is exact to the split second. But not in the specific
rounding part, really.Here is what I do: Enter, for example, 3:55:59 (3.5559 ENTER, convert to decimals with the built in function (Time, HMS->) add 1:59:20 (1.5920) by converting and convert the result back (->HMS) I get 5:11:19 which is correct. So, I don’t get that
How is the original question/answer _4_ seconds off? Isn't 0.015999999988 almost 0.02 ...which is 2 minutes?You are right. Got a bit dizzy, obviously ;-) Thank you for putting me back into the picture :-)
Sysop: | Keyop |
---|---|
Location: | Huddersfield, West Yorkshire, UK |
Users: | 355 |
Nodes: | 16 (3 / 13) |
Uptime: | 04:15:34 |
Calls: | 7,656 |
Calls today: | 8 |
Files: | 12,812 |
Messages: | 5,700,752 |