# The Relevance-Based Prediction Engine This is the knowledge base to support the Relevance-Based Prediction Engine (RBP Engine), a quantitative prediction package for data analytics across industries. Leverage relevance-based prediction and its insights within your data science environment of choice. Quickly prototype and evaluate exhaustive prediction models and access the power of Relevance-Based Prediction. ## What is the RBP Engine? The RBP Engine is a package library of functions available for native use within macOS (arm64), Linux (arm64, Intel/AMD), and Windows (Intel/AMD) systems. You can integrate RBP into your Python, R, Matlab, or systems workflows. ## Why use the RBP Engine? Data analysts may find the RBP Engine valuable as part of their daily workflow to - Generate task-specific forecast from your data. - See advance guidance on the reliability of specific predictions - See which observations are important in forming the prediction - Identify attributes and performance variables that matter most for each prediction ## Conceptual Overview Over the past 230 years, mathematicians and researchers have advanced the science of data-driven prediction by developing models that range from simple linear regressions to complex neural networks that power modern generative AI systems. Unfortunately, all these models face an inescapable and limiting tradeoff: simple models cannot capture the complexity of the real world, while complex models are notoriously opaque and prone to making dramatic errors that cannot be detected in advance.   Relevance-based prediction (RBP) is a new and fundamentally different way to form data-driven predictions. Unlike models which rely on large numbers of parameters that are difficult or impossible to interpret, RBP is model-free and forms a prediction as a simple weighted average of prior outcomes for the variable we wish to predict. As a result, the logic of RBP is completely transparent, and it is aligned with the way people naturally think about extrapolating from experience to predict future outcomes. The key innovations that power RBP define a rigorous method for determining the optimal prediction weights for each individual prediction task.   A fundamental limitation of any model-based approach to prediction is an overreliance on averages. One must both select the correct model and identify optimal values for its many parameters. Both these jobs are done by evaluating the performance of a model on average across all prediction tasks. These averages mask the fact that some predictions could be very badeven while others are very good. Models always give a prediction, but they provide hardly any information about how the prediction was formed, and they do not reveal in advance whether the prediction for a specific task is reliable. If the prediction circumstances are not well represented by the training sample, the prediction may be unreliable, but it would be impossible to knowthis until we observe the outcome after we may have already made the mistake.   RBP overcomes the limitations of both linear regression analysis and AI models. Like linear regression analysis, RBP has a strong theoretical foundation, but unlike linear regression analysis, it easily accommodates complex dynamics. And RBP extracts as much information from complex datasets as AI models, but unlike AI models it is fully transparent. The transparency of RBP facilitates a variety of important advantages. It automatically adapts to new prediction circumstances. It gives advance notice of the unique reliability of each prediction before it is made. And it is inherently resilient to missing data and data errors. These advantages are not available with model-based prediction. Finally, owing to a range of mathematical convergences, RBP provides a lens for reinterpreting classical statistics.   ## How RBP Works  RBP forms a prediction as a weighted average of previously observed outcomes in which the weights are based on a precise and theoretically justified statistic called relevance. RBP has three key features: relevance, fit, and grid prediction.   Relevance measures the importance of an observation to a prediction. It is composed of similarity and informativeness, which are both measured using a statistic called the Mahalanobisdistance. The Mahalanobis distance is an elegant measure of the distance between values of a set of variables, such as physical attributes or performance metrics, that accounts not only for how the corresponding values compare independently but also how they interact with each other. Similarity measures the multivariate distance of a prior occurrence from current circumstances. Informativeness measures the multivariate distance of a prior occurrence from average. An important tenet of information theory is that unusual events warrant close attention because they contain more information than typical events. The bottom line is that data observations that are like current circumstances but different from average are more relevant than those that are not.   It is important to note that RBP converges to linear regression analysis in a special case. When relevance weights are determined from all available variables and are applied to every past observation in a sample, the weighted average of outcomes gives exactly the same prediction as a linear regression model. From this equivalence, it is clear to see that the commonly used approach of linear regression analysis exhibits both desirable and undesirable behaviors. It places large positive weights on relevant past experiences, implying that similar outcomes may occur, which makes sense. However, it places large negative weights on the least relevant past experiences, reasoning that the opposite of the most opposite experiences will occur, which usually is not helpful. Predicting the opposite-of-opposites only works for relationships that are static and symmetric, which is rarely the case in practice. RBP is based on the premise that we can form a better prediction by focusing on the most informative and reliable subsets of relevant observations. Doing so requires a new measure of fit.   Fit measures the extent to which there are useful patterns in a dataset. It is measured as the average standardized alignment between the relevance weights of a given prediction task and the outcomes for every pair of observations that go into it. Fit is determined before a prediction is made. It, therefore, gives advance guidance about the reliability of that specific prediction. Tasks for which relevant observations contain valuable patterns of alignment will tend to yield more reliable predictions than tasks for which the relevant observations contain mostly random noise. No other prediction technique can give advance notice of an individual prediction’s reliability. The ubiquitous R-squared statistic provides information about a model’s quality, but it is based on some good predictions, some bad predictions, and some so-so predictions. It is a grand average. By contrast, fit gives advance notice of the quality of each individual prediction, distinct from the next. It is analogous to an R-squared for an individual prediction. In fact, we have proven that fit is a mathematically exact decomposition of R-squared in the case of linear regressionanalysis.  We wish to again emphasize that the insights given by fit for individual predictions are unobtainable without RBP.  The final feature of RBP is grid prediction. It uses fit to precisely blend the predictions that result from different combinations of observations and predictive variables. Crucially, the blend places greater emphasis on the observations and variables that are most useful for an individual prediction task. The columns in the grid represent different combinations of predictive variables, and the rows represent different subsamples of observations based on various relevance thresholds. Each cell in the grid provides a prediction along with a measure of its reliability, based on fit. The grid forms a composite prediction as a reliability-weighted average of the predictions from all the individual cells. This process diversifies the prediction across many calibrations in a way that bends toward those that are more reliable. The prediction grid allows us to describe the final prediction in terms of events that actually occurred.   The prediction grid also yields a comprehensive measure of how important each variable is to the reliability of a specific prediction. This measure is called relevance-based importance (RBI). It is computed as the average fit of the cells that include a given variable minus the average fit of the cells that omit that variable, properly scaled. Conventional models rely on t-statistics which only measure a variable’s marginal importance. RBI, by contrast, captures a variable’s total importance. It also captures conditional relationships which t-statistics fail to address. And unlike the Shapley value, which is the accepted standard for assessing variable importance in AI models, RBI accounts for the reliability of individual predictions.   The prediction grid makes RBP more resilient to missing data than model-based approaches. It preserves more information than models when there is missing data, and it accounts for the relative importance of the missing data in the formation of the prediction and in its assessment of the prediction’s reliability.   ## Transparency and Power  As mentioned earlier, a key feature of RBP is that it is model-free. Unlike a regression model or a neural network, it has no parameters. It forms a prediction as a weighted average of observed outcomes in which the weights are based on relevance. This feature enables full transparency into how each observation informs each prediction and how each predictive variable contributes to the reliability of each prediction. And it reveals each prediction’s reliability in advance of making the prediction. Model-based approaches only give information about average reliability and the average influence of observations and variables. The problem with averages is that they provide misleading guidance in many of the most important situations.   Finally, it is important to grasp how RBP can extract as much information from a complex dataset as a neural network, but more efficiently and with full transparency. A neural network uses pretrained nested layers of averages and non-linear transformations to address the potential complexity of every possible prediction task. As noted earlier, this process is guided by fixed parameters that are selected based on average efficacy and typically require immense volumes of data and intense computation to configure. RBP instead proceeds task by task. It therefore only needs to address the complexity of one prediction task at a time. Rather than generate a vast number of parameters whose features are obscured by repetitive transformations, RBP considers a vast number of combinations of observations and predictive variables, identifying those that are most important to the task at hand. For example, if RBP uses 20 predictive variables and 10 subsamples of observations, it will produce a grid of more than 10 million cells for each prediction task. It is the prediction-specific nature of RBP, together with the consideration of so many combinations of observations and variables, that gives RBP so much power and transparency.  ## Conclusion  RBP is a powerful and fully transparent prediction technology that rests on solid theoretical underpinnings including information theory, the Central Limit Theorem, and the Mahalanobis distance, and it is further supported by several remarkable mathematical convergences. It has been tested in a variety of real-world applications in finance, elections, medicine, and sports. Some of these applications are described in the publications listed below.  RBP is distributed to collegiate and professional sports teams by Cambridge Sports Analytics.