The buttons are being reordered because they are no longer children of the DockPanel. Instead the ItemsControl is now ordering the buttons. I would suggest using a StackPanel inside of a ScrollViewer and just inserting the new button as the first child.
<ScrollViewer>
<StackPanel Name="spButtons">
<Button Name="button4">Last added, on top</Button>
<Button Name="button3">Next to last</Button>
<Button Name="button2">Second</Button>
<Button Name="button1">First added, on bottom</Button>
</StackPanel>
</ScrollViewer>
Then you can add new buttons on top like this:
spButtons.Children.Insert(0, buttonToAdd)
