《Elements of Statistical Learning Solution Book 2》.pdf

Consider a regression problem with inputs and outputs , and a parameterized model () to be fit by least squares. Show that if there are observations with tied or identical values of x, then the fit can be obtained from a reduced weighted least squares problem. For known heteroskedasticity (e.g., grouped data with known group sizes), use weighted least squares (WLS) to obtain efficient unbiased esti- mates. Fig.1 explains “Observations with tied or identical values of x”. In • 7 • • 6 • • 5 • • y 4 • • 3 • • 2 • 10 15 20 25 x WLS.01 Figure 1: Observations with tied the textbook, section 2.7.1 also explain this problem. 1 If there are multiple observation pairs = 1 2 at each value of , the risk is limited as follows: argmin ∑ ∑( ( ) − ) = argmin ∑( ( ) − 2 ∑ ( )