-
I am in need to aggregate my stream on multiple columns, in SQL I would do something similar to Is this to be expected or did I made a mistake in my flow that prevent it to work with one single node? The previous node is a window node as I would like to compute avg and min from a 10 minutes stream. |
Beta Was this translation helpful? Give feedback.
Replies: 2 comments
-
@Zap123 Initially, the Graph API was based on SQL concepts. However, we intentionally moved away from this design in later versions because it was counterintuitive for users. To address this, we made each operator compatible with both scalar values and collections so that users can actually connect any nodes. This change means that when an aggregation function runs, the new aggregated value is simply appended to the existing data. Because the original collection information is preserved, you can perform another aggregation in a subsequent node. |
Beta Was this translation helpful? Give feedback.
-
Thanks @ngjaying this clarifies it. I like the Graph API a lot but unfortunately I couldn't find this detail in the documentation and I had to discover it. |
Beta Was this translation helpful? Give feedback.
@Zap123 Initially, the Graph API was based on SQL concepts. However, we intentionally moved away from this design in later versions because it was counterintuitive for users. To address this, we made each operator compatible with both scalar values and collections so that users can actually connect any nodes.
This change means that when an aggregation function runs, the new aggregated value is simply appended to the existing data. Because the original collection information is preserved, you can perform another aggregation in a subsequent node.