DATA SCIENCE: JOURNAL OF COMPUTING AND APPLIED INFORMATICS (JoCAI)

The two foremost aims of classical regression are to assess the structure and magnitude of the relationship between variables. Despite the aforementioned benefits, unlike classical regression, which only offers a point estimate and a confidence interval, Bayesian regression offers the whole spectrum of inferential solutions. The results of this study demonstrate the Bayesian approach's suitability for regression tasks and its advantage in accounting for additional a priori data, which often strengthens studies. Using data from Boston Housing from UCI ML Repository, this study proves that the prior distributions have the benefit of producing analytical, closed-form conclusions, which eliminates the need to use numerical techniques like Markov Chain Monte Carlo (MCMC). Second, software implementations are offered together with formulas for the posterior outcomes that are supplied, clarified, and shown. The assumptions supporting the suggested approach are evaluated in the third step using Bayesian tools. Prior elicitation, posterior calculation, and robustness to prior uncertainty and model sufficiency are the three processes that are essential to Bayesian inference. The findings confirm that Bayesian models provide a full posterior distribution over model parameters, which can be used to quantify uncertainty and make more informed predictions and can also incorporate regularization techniques to reduce overfitting and improve generalization performance

The two foremost aims of classical regression are to assess the structure and magnitude of the relationship between variables. Despite the aforementioned benefits, unlike classical regression, which only offers a point estimate and a confidence interval, Bayesian regression offers the whole spectrum of inferential solutions. The results of this study demonstrate the Bayesian approach's suitability for regression tasks and its advantage in accounting for additional a priori data, which often strengthens studies. Using data from Boston Housing from UCI ML Repository, this study proves that the prior distributions have the benefit of producing analytical, closed-form conclusions, which eliminates the need to use numerical techniques like Markov Chain Monte Carlo (MCMC). Second, software implementations are offered together with formulas for the posterior outcomes that are supplied, clarified, and shown. The assumptions supporting the suggested approach are evaluated in the third step using Bayesian tools. Prior elicitation, posterior calculation, and robustness to prior uncertainty and model sufficiency are the three processes that are essential to Bayesian inference. The findings confirm that Bayesian models provide a full posterior distribution over model parameters, which can be used to quantify uncertainty and make more informed predictions and can also incorporate regularization techniques to reduce overfitting and improve generalization performance.

Keywords:
Bayesian linear regression Bayesian inference Model Housing Price Email: harry.patria@sbm-itb.ac.id

Introduction
The goal of science is to comprehend phenomena and systems to predict and control their advancement. Models, which are advanced mathematical or algorithmically representations in the language of quantitative sciences, play a vital role in the scientific process of knowledge elaboration found in a wide range of industrial applications such as energy [1,2,3], finance [4,5,6,7,8].
Renting and purchasing houses are becoming more popular as cities become more crowded. As a result, predicting and modeling house prices is a crucial subject in a way for determining a more robust method of calculating house prices [9], [10]. Furthermore, this is going to help both sellers and buyers find the best price for their homes [11], strengthening the economy during a recent global recession. This result, which accurately reflects market price become a hot topic in the real estate sector.
This study aims to shed some light on Bayesian linear regression used to model house prices. One advantage of using a Bayesian model for predicting housing prices is that it allows for the incorporation of prior knowledge or belief about model parameters. This can be especially useful in situations where there is limited data available or the data is noisy. Secondly, Bayesian models also allow for the incorporation of regularization techniques, such as shrinkage priors, which can help to reduce overfitting and improve the generalization performance of the model. In addition, Bayesian models can naturally handle missing data, by using appropriate probability models to represent the missing data process.
Another potential advantage of using a Bayesian model for predicting housing prices is that it provides a full posterior distribution over model parameters, which can be used to quantify uncertainty and make more informed predictions. This can be especially useful in a decision-making context, as it allows for the explicit consideration of model uncertainty in decision-making processes. Overall, Bayesian models can be a powerful tool for predicting housing prices and may offer some advantages over traditional models such as multiple linear regression. However, it's important to carefully consider the specific needs and characteristics of a given prediction problem and choose the most appropriate modeling approach accordingly.
The advice is organized around the steps of a Bayesian approach. Section 1 outlines the background and purpose. Section 2 describes the literature review while section 3 describes the elicitation of the dataset and method used in this study such as a prior distribution and the estimated posterior distribution. The distributions of the regression parameters, the regression function, and the model, as well as their estimates and uncertainties, are given and explained in Section 4. Lastly, section 5 summarizes the findings and discussion in a way to generalize the findings and application.

Literature Review
The literature makes an effort to extract practical knowledge from historical real estate market data. To find models that are helpful to home buyers and sellers, machine learning techniques are widely used [9,10,11]. Phan [9] investigated historical Australian real estate transactions, showing the significant price disparity between homes in Melbourne's most costly and least expensive suburbs. Additionally, tests show that the Stepwise and Support Vector Machine combo, which is based on mean squared error assessment, is a competitive strategy.
In China, Yu et al. [10] applied multiple prediction models based on deep learning to determine the current real estate data. The results provide a more precise prediction of the housing price or its changing trend in the future. This will allow us to analyze the effects of various factors on housing prices. In light of machine learning, Vineeth et al. [11] scrutinized the house price using machine learning algorithms, ranging from simple linear regression (SLR), Multiple linear regression (MLR), and Neural Networks (NN).
Bayesian regression methods are extremely powerful because they provide us with an entire distribution over the regression parameters rather than just point estimates [12,13,14]. This can be understood as learning not just one model, but an entire family of models and assigning different weights to them based on their likelihood of being correct. Because this weight distribution is affected by the observed data, Bayesian methods can provide us with an uncertainty quantification of our predictions that represents what the model was able to learn from the data [15,16,17,18,19]. The uncertainty measure could be, for example, the standard deviation of all model predictions, which point estimators do not provide by default. Not only transforming AI into more understandable but also changing a paradigm about probability and uncertainty to augment the analytical modeling better.
Bayesian linear regression is a useful tool for forecasting problems, and there are several reasons why it might be preferred over other machine learning techniques such as simple or multiple linear regression or neural networks. Bayesian approaches allow for the incorporation of prior knowledge about model parameters, and can also handle regularization and missing data. In addition, Bayesian linear regression provides a full posterior distribution over model parameters, which can be used to quantify uncertainty and make more informed predictions [15,16,17]. Overall, Bayesian linear regression is a flexible and powerful tool that may be well-suited for certain types of data and modeling scenarios.

Data Collection
This study used Boston Housing provided by the UCI repository. Each of the 506 entries provides aggregate information about 14 characteristics of residences from different Boston suburbs, and the data was collected in 1978. The detail of descriptive statistics of the variables can be found in Table 1.  The preliminary analysis using a correlation heatmap describes substantial findings found in Figure 1. First, the average room (RM) and median value of the house (MEDV) are highly correlated, with a score of 0.70. On the other hand, the results found a highly negative correlation between % lower status of the population % lower status of the population (LSAT) and the aforementioned MEDV, showing a score of -0.61. Since this study merely focuses on house features, therefore, RM will be used in predicting MEDV than LSTAT. In particular, this study scrutinizes the plausible connection between house features and their estimated price by following the Bayesian approach.

Bayesian Linear Regression
Bayesian linear regression is a statistical approach that is often used to model the relationship between a dependent variable and one or more independent variables. It is based on the idea of using Bayes' theorem to update our belief about the values of the model parameters given the data that we observe. This approach has been widely used in various fields for a variety of purposes, including prediction, estimation, and inference.
One notable application of Bayesian linear regression is in the field of finance, where it has been used to model a variety of financial data including stock prices, exchange rates, and interest rates [20]. Bayesian linear regression has also been used in the field of engineering, where it has been applied to problems such as surface roughness prediction [21] and short-term travel speed.
This section outlines linear regression using probability distributions rather than point estimates in light of a Bayesian perspective. The response, , is assumed to be drawn from a probability distribution rather than being estimated as a single value. The Bayesian Linear Regression model with a response sampled from a normal distribution is illustrated by Equation 1 below: A notable symbol of y is defined as an output of a normal (Gaussian) distribution with a mean and variance. For linear regression, the mean is calculated by multiplying the weight matrix by the predictor matrix. The variance is equal to the standard deviation squared (multiplied by the Identity matrix because this is a multi-dimensional formulation of the model). The goal of Bayesian Linear Regression is not to find a single "best" value for the model parameters, but rather to determine the posterior distribution for the model parameters. Not only is the response generated from a probability distribution, but again the model parameters are also assumed to be generated from a distribution. The posterior probability of the model parameters is completely reliant on the training inputs and outputs:

Posterior Probability Distribution
Given the inputs and outputs, P(|y, X) represents the posterior probability distribution of the model parameters. This is equal to the prior probability of the parameters multiplied by the likelihood of the data, P(y|, X), and divided by a normalization constant. This is a straightforward formulation of the Bayes Theorem, which serves as the cornerstone of Bayesian inference. As opposed to OLS, our model's parameters have a posterior distribution that is proportional to the likelihood of the data times the prior probability of the parameters. In terms of priors, we can integrate domain knowledge or an educated estimate as to what the model parameters should be in our model, unlike the frequentist approach, which presupposes that all information about the parameters comes from the data. In this vein, we can utilize non-informative priors for the parameters, like a normal distribution, if we do not already have any estimations.

Result and Discussion
The posterior distributions of the model parameters are approximated in Figure 2 and Figure 3. According to the findings, Bayesian models offer a complete posterior distribution of model parameters, enabling quantification of uncertainty and more informed predictions. They also incorporate regularization techniques to prevent overfitting and enhance generalization performance. The model produces the stable outcomes of 1000 MCMC steps, which means that 1000 steps were taken from the posterior distribution by the method.  To demonstrate the effect of the number of data points in the model, author used two attempts using a different number of data points (1,000 and 10,000), and the resulting fits are shown consecutively in Figures 4 and 5. From the two findings in Figure 4 and Figure 5, it can be concluded that when fewer data points lead to adjustments significantly, indicating more uncertainty in the model. Thus, we also obtain a distribution rather than a single result when using our Bayesian Linear Model to forecast the outcome for a single data point. When it comes to posterior, a distribution of potential model parameters depending on the data and the prior is the outcome of performing Bayesian Linear Regression. With fewer data points, the posterior distribution will be more dispersed, allowing us to measure how doubtful we are about the model.

Conclusions
Based on the findings, it can be concluded that Bayesian models are a useful tool for predicting and modeling various types of data. These models provide a full posterior distribution over model parameters, which allows for the quantification of uncertainty and the ability to make more informed predictions. Additionally, Bayesian models can incorporate regularization techniques to reduce overfitting and improve generalization performance. Overall, the use of Bayesian models can be a valuable approach for a wide range of prediction and modeling tasks.
In cases where we have limited data or prior knowledge that we want to incorporate into our model, the Bayesian Linear Regression approach can incorporate prior knowledge while also displaying our uncertainty. The Bayesian framework is reflected in Bayesian Linear Regression: we form an initial estimate and improve it as more data is gathered. The Bayesian approach is a sensible way to perceive the world, and Bayesian Inference can be a valuable counterpoint to frequentism. The goal of data science is to identify the best tool for the job, not to take sides.
The forthcoming study could include a discussion of potential future studies in the conclusion section that further demonstrate the value of the Bayesian approach and how it can be a useful counterpoint to frequentism. This could include outlining specific research questions or directions that could be pursued in future work, and explaining how these studies would help to further illustrate the benefits of using the Bayesian approach in various contexts. Additionally, the authors could consider providing concrete examples of challenges or problems that the Bayesian approach is particularly well-suited to addressing, and explaining how future studies could help to address these challenges and contribute to a better understanding of the usefulness of the Bayesian approach.".