What would you think about something like local linear least squares (where it uses linear regression but locally weights the linear regression so that it is designed to fit points closer to the target)?

Is it learning patterns in the data, or is it just a glorified lookup table?