feat: Allow log with non-integer base on decimals#19372
feat: Allow log with non-integer base on decimals#19372Jefffrey merged 9 commits intoapache:mainfrom
Conversation
ece37a4 to
683b1cc
Compare
|
cc: @alamb |
datafusion/functions/src/math/log.rs
Outdated
| /// Returns error if base is invalid | ||
| fn log_decimal128(value: i128, scale: i8, base: f64) -> Result<f64, ArrowError> { | ||
| if !base.is_finite() || base.trunc() != base { | ||
| if !base.is_finite() { |
There was a problem hiding this comment.
I think we need to check some of these other error conditions such as base being non-finite or base being less than 1.0, as I don't think they error on float version
There was a problem hiding this comment.
check the recent changes
Using a single trait and unified the previous functions !
cc: @Jefffrey
f0b96f7 to
160c98e
Compare
datafusion/functions/src/math/log.rs
Outdated
| fn unscale_decimal_value<T: ToPrimitive>(value: &T, scale: i8) -> Option<u128> { | ||
| let value_u128 = value.to_u128()?; |
There was a problem hiding this comment.
I'm not sure about this choice to universally convert to u128, even for decimal32 & decimal64
There was a problem hiding this comment.
Valid ! It's overkill
|
|
||
| #[test] | ||
| fn test_log_decimal128_wrong_base() { | ||
| fn test_log_decimal128_invalid_base() { |
There was a problem hiding this comment.
Could we move these tests to SLTs
f3266bd to
cb29663
Compare
|
cc: @Jefffrey |
|
I must admit I am very lost now; it seems this PR has removed all the logic for computing log natively on decimal and now we manually convert to float? |
Apologies ! I misunderstood your feedback about u128 and overcorrected by removing the native decimal logic entirely. |
Co-authored-by: Jeffrey Vo <jeffrey.vo.australia@gmail.com>
Co-authored-by: Jeffrey Vo <jeffrey.vo.australia@gmail.com>
|
Thanks @Yuvraj-cyborg |
Which issue does this PR close?
Closes #19347
Rationale for this change
Native decimal log() (added in #17023) only supports integer bases. Non-integer bases like log(2.5, x) error out, which is a regression from the previous float-based implementation.
What changes are included in this PR?
Changes :
Refactoring:
Large Decimal256 values that don't fit in i128 now work via f64 fallback.
Are these changes tested?
Yes:
Are there any user-facing changes?
Yes: