www.sdlsuite.com Forum Index www.sdlsuite.com
Discussion of technical aspects of the SDL Component Suite
 
 FAQFAQ   SearchSearch   MemberlistMemberlist   UsergroupsUsergroups   RegisterRegister 
 ProfileProfile   Log in to check your private messagesLog in to check your private messages   Log inLog in 

TNumLab calculation wrong

 
Post new topic   Reply to topic    www.sdlsuite.com Forum Index -> GuiPack
View previous topic :: View next topic  
Author Message
rblum



Joined: 18 Sep 2006
Posts: 4

PostPosted: Tue Apr 05, 2011 4:30 pm    Post subject: TNumLab calculation wrong Reply with quote

Hallo,

if I feed TNumLab with the value: 0.99999 (five digits of 9) the displayed value is 0 59' 59.96" which is alright.
But with 0.999999 (6 digits of 9) the following is the result:
0 59' 60.00 which is clearly wrong. The 60 seconds are 1 minute, so the value should propagate and add to the 59' whose value will be 60' which again propagate to the degree, resulting in 1.

properties:
DisplayType = dtDegMinSec
Precision = 2

Kind regards
Blum
Back to top
View user's profile Send private message Send e-mail
hlohningadmin
Site Admin


Joined: 01 Jan 1970
Posts: 9

PostPosted: Wed Apr 06, 2011 8:57 am    Post subject: Reply with quote

Thanks for the bug report. It seems that this is caused by a round-off error. We'll fix that in the next release.

Regards,

Hans
_________________
---
Hans Lohninger
Software Development Lohninger
http://www.lohninger.com/
Back to top
View user's profile Send private message
rblum



Joined: 18 Sep 2006
Posts: 4

PostPosted: Fri May 27, 2011 3:21 pm    Post subject: Reply with quote

Hallo Hans,

my experiments show, that the decimal fraction of the last second should be rounded usually and should be truncated if = 59.

Regards

Rolf
Back to top
View user's profile Send private message Send e-mail
hlohning



Joined: 08 Sep 2006
Posts: 277
Location: Austria

PostPosted: Thu Jun 02, 2011 10:15 am    Post subject: Reply with quote

Hi Rolf,

this bug is actually a bug in DegreeToStr (unit sdl_math1). It is already solved, the new code will be available with the next release (in July).

Regards,

Hans
_________________
Hans Lohninger
Software Development Lohninger
http://www.lohninger.com/
Back to top
View user's profile Send private message Visit poster's website
Display posts from previous:   
Post new topic   Reply to topic    www.sdlsuite.com Forum Index -> GuiPack All times are GMT + 1 Hour
Page 1 of 1

 
Jump to:  
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum


Powered by phpBB © 2001, 2005 phpBB Group