Yes, speed falls - you drop down the efficiency curve quite quickly, not as quick as the tail-off at the top end but it is significant. You lose much more than just the straight linear ratio between wheel diameters and it can be a cascading reduction as you are left with rapidly reducing (EDIT)...