If I were in this situation I would start by getting the models setup
with their relationships, using the scaffolds to build simple CRUD
interfaces and provide tests to make sure that is working ok. Once
that is going then you can build on top the UI that you want, a bit at
a time, knowing that the underlying data is ok. Later you can remove
any bits of the scaffold that are redundant.
Others would tackle this differently of course.