Skip to content

Commit f7430b1

Browse files
committed
Add Decimal32/64 in Arrow C++ as a fundable project
1 parent f8b629c commit f7430b1

File tree

4 files changed

+70
-0
lines changed

4 files changed

+70
-0
lines changed
Lines changed: 40 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,40 @@
1+
#### Overview
2+
3+
Apache Arrow is the universal columnar format and multi-language toolbox for fast data interchange and in-memory analytics.
4+
5+
Fixed-width decimal data in Arrow is usually represented the Decimal128 data type.
6+
This data type has non-trivial memory costs (16 bytes per value) and computational costs (operations on 128-bit integers must be emulated on most if not all architectures).
7+
8+
Arrow recently gained Decimal32 and Decimal64 data types which, as their names suggest, encode fixed-width decimal data more compactly.
9+
Decimal32 (resp. Decimal64) is able to represent up to 9 (resp. 18) decimal digits of precision, which is sufficient in many applications.
10+
11+
However, while basic support is present, Decimal32 and Decimal64 are not universally supported by all Arrow components.
12+
13+
We propose to finish implementing support for Decimal32 and Decimal64 types in all components of Arrow C++:
14+
15+
* scalar compute kernels:
16+
- `abs`
17+
- `round`
18+
- `is_in`, `index_in`
19+
- `coalesce`
20+
- `min_element_wise`, `max_element_wise`
21+
22+
* vector compute kernels:
23+
- `dictionary_encode`, `unique`, `value_counts`
24+
- `pairwise_diff`
25+
- `select_k_unstable`
26+
- `replace_with_mask`
27+
- `fill_null_forward`, `fill_null_backward`
28+
29+
* aggregate compute kernels:
30+
- `sum`, `mean`, `mode`, `tdigest`
31+
- `first`, `last`, `min`, `max`
32+
- `index`
33+
34+
* CSV reader and writer
35+
36+
* ORC reader and writer
37+
38+
Funders can decide to fund the entire package, or choose the components they are interested in.
39+
40+
##### Are you interested in this project? Either entirely or partially, contact us for more information on how to help us fund it

src/components/fundable/projectsDetails.ts

Lines changed: 12 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -99,6 +99,18 @@ export const fundableProjectsDetails = {
9999
currentNbOfFunders: 0,
100100
currentFundingPercentage: 0,
101101
repoLink: "https://github.com/apache/arrow"
102+
},
103+
{
104+
category: "Apache Arrow and Parquet",
105+
title: "Complete Decimal32 / Decimal64 support in Arrow C++",
106+
pageName: "Decimal32InApacheArrow",
107+
shortDescription: "Decimal32 and Decimal64 are more compact and computationally more efficient data types than the standard Decimal128.",
108+
description: Decimal32InArrowCpp,
109+
price: "TBD",
110+
maxNbOfFunders: 2,
111+
currentNbOfFunders: 0,
112+
currentFundingPercentage: 0,
113+
repoLink: "https://github.com/apache/arrow"
102114
}
103115
]
104116

Lines changed: 9 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,9 @@
1+
import useDocusaurusContext from '@docusaurus/useDocusaurusContext';
2+
import GetAQuotePage from '@site/src/components/fundable/GetAQuotePage';
3+
4+
export default function FundablePage() {
5+
const { siteConfig } = useDocusaurusContext();
6+
return (
7+
<GetAQuotePage/>
8+
);
9+
}
Lines changed: 9 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,9 @@
1+
import useDocusaurusContext from '@docusaurus/useDocusaurusContext';
2+
import LargeProjectCardPage from '@site/src/components/fundable/LargeProjectCardPage';
3+
4+
export default function FundablePage() {
5+
const { siteConfig } = useDocusaurusContext();
6+
return (
7+
<LargeProjectCardPage/>
8+
);
9+
}

0 commit comments

Comments
 (0)