With DimensionalData everything is strongly typed, and we've adopted that for Dataset and InferenceData as well, where the underlying storage is a NamedTuple. One of the consequences of this is that if a user adds a variable to a group, then the resulting InferenceData now has a new type, and so there are frequent delays due to JIT compiling.
It would be nice if we could figure out a workaround for this. DataFrames seems to do so by having the underling storage be an OrderedCollections.LittleDict. I'm guessing DataFrames maintains efficiency when operating on columns/rows by using function barriers everywhere. We could do something similar. This would also allow InferenceData and Dataset to be modified in-place. Type inferrability for efficiency is likely only critical when operating on variables themselves.
With DimensionalData everything is strongly typed, and we've adopted that for
DatasetandInferenceDataas well, where the underlying storage is aNamedTuple. One of the consequences of this is that if a user adds a variable to a group, then the resultingInferenceDatanow has a new type, and so there are frequent delays due to JIT compiling.It would be nice if we could figure out a workaround for this. DataFrames seems to do so by having the underling storage be an
OrderedCollections.LittleDict. I'm guessing DataFrames maintains efficiency when operating on columns/rows by using function barriers everywhere. We could do something similar. This would also allowInferenceDataandDatasetto be modified in-place. Type inferrability for efficiency is likely only critical when operating on variables themselves.