Could be simplified to:

int closest = inputs.First(x => Math.Abs(x - inputs.Average()) == inputs.Min(y => Math.Abs(y - inputs.Average())));

but I don't like to perform the same calculation twice, so I'd be more comfortable with:

double avg = inputs.Average();
int closest = inputs.First(x => Math.Abs(x - avg) == inputs.Min(y => Math.Abs(y - avg)));