Better Modularization for Rails 2.0

I've written an article suggesting some radical changes to the Rails
directory structure for Rails 2.0 and am curious to know the core
team's thoughts about it:

http://wiseheartdesign.com/2007/6/15/better-modularization-for-rails-2-0/

Nice article, short and to the point.

While the benefits are certainly recognizable, I, for one, would find slightly shocking to have a directory like this:

blog/
    _config.rb
    _routes.rb
    blog_controller.rb
    blog_helper.rb

    blog_index.rhtml
    blog_archive.rhtml
    blog_post.rhtml
    comment.rb
    comment_migrations.rb
    comments_controller.rb
    comments_helper.rb
    comments_view.rhtml
    post.rb

    post_migrations.rb

I’ve been spoiled by the clean separation in Rails and I’d hate to go back to spaghetti directory structure. I imagine that most of its users would feel the same, too.

I've written an article suggesting some radical changes to the Rails
directory structure for Rails 2.0 and am curious to know the core
team's thoughts about it:

Thanks for thinking about this John, but I don't see Rails adopting
something along those lines right now. We can definitely do more to
improve high-level reuse for the sub-set of users who need it, but it
probably won't be setting the tone quite as dramatically as this. I
much prefer to have things divided by type when I'm just working on a
single application. And anything that includes prefixing underscores
is a visual non-starter for me ;).

Some of my larger plugins tend to look like that, though. It feels
wrong, I want to add directories to separate models and controllers.
Though ultimately, textmate makes it so I don't have to care that much
about the actual directory structure.

I like the writeup, but I don't see it as a big enough benefit to
justify the pain of changing it. Both structures have their benefits.
If one was clearly superior then I say damn the pain and change it,
but in this case it seems more like an arbitrary choice. Rails chose
the MVC structure and Django chose the component structure. In a way
it reflects the philosophies of the two frameworks.

I like this structure much better than Rails' components though.
Maybe this type of structure could be used for a lighter-weight
version of components for Rails 2.0. At first glance it would appear
ugly to have two different kinds of file organization, but maybe
there's a strong use case for lightweight components? At any rate,
it's great to see this kind of fresh thinking.

+1

"_foo" == :frowning:

b

DHH wrote:

Thanks for thinking about this John, but I don't see Rails adopting
something along those lines right now. We can definitely do more to
improve high-level reuse for the sub-set of users who need it, but it
probably won't be setting the tone quite as dramatically as this. I
much prefer to have things divided by type when I'm just working on a
single application.

Help me out here. I'm not sure I understand why type is better.
Setting the modularization argument aside grouping your application
into logical parts seems like a win to me.

And anything that includes prefixing underscores
is a visual non-starter for me ;).

Humor me for a minute here. The underscores aren't the critical part
of the idea.

The thing that immediately struck me about this proposal was the
interdependencies between the different parts of the application. The
example in the original post showed several (of what I presume are) models
in the app/blog subdirectory, but I'd expect that the models and controllers
in that subdir also require the model in the app/account subdirectory. If
you wanted to give someone else the app/blog subdirectory to use in their
application, then some degree of scaffolding or similar would be required in
the user model to make it work. I'm curious if django has a solution to
this problem? Usually, in my applications, there would almost always be a
single directory that had all of the models and controllers, since there's
almost always a dependency chain that links all of them together.

As I understand it, the purpose of the rearrangement would be to support
making it trivial to carve out subsets of a Rails application for re-use,
but in my experience, the directory structure is the *last* thing you need
to be worrying about. There are many, many more important things to decide
on before concerning yourself with how many different directories you need to
copy files out of.

If your goal is only to allow people to structure plugins in a more
Rails-like manner, rather than assisting so much in the *extraction* of the
plugins, then I would suggest that adding
vendor/plugins/<plugin_name>/app?/{models,controllers,helpers,views} to the
search paths would achieve that aim without requiring what is a fairly
fundamental change to the way in which all Rails applications are
structured.

For myself, I think that the proposed change is an aesthetic loss, but that
may simply be familiarity with the existing layout talking.

- Matt

I've written an article suggesting some radical changes to the Rails
directory structure for Rails 2.0 and am curious to know the core
team's thoughts about it:

http://wiseheartdesign.com/2007/6/15/better-modularization-for-rails-2-0/

Leaving aside the asthetic issues, and the high cost to tooling and
books of changing the application layout, I'm curious about the whole
rationale.

"One of the primary hinderances of the present Rails architecture to
modularization is the directory structure."

Is this even true? We have configurable load paths, controller
paths, and view paths. I simply can't see how the directory structure
is even on the list of the top ten problems to 'modularization' of a
rails application.

Is this even true?

I’d submit that, yes, it is. The best way that I’ve found of sharing entire “slices” of an application, is simply pluginize the whole thing. Which is rather bothersome. But, in that case, I simply have an svn:external (or in some cases just a symlink) pointing to the plugin. To do that without pluginizing, using the current directory structure, is impractical.

That being said though, I don’t think thats enough reason to justify such a change. Others have already mentioned the appropriate reasons as to why its not.

Tyler

I agree as well. It's only ONE problem, but it's a big one.
Distribution, upgrades, and even just seeing what files came from
where. Generators don't solve it, and plugins just seem wrong. The
closest thing would be Components, but of course they have their own
issues.

The other problems may or may not be more significant. Granted, you
can't rip out the average controller from an arbitrary app and re-use
it, but there's no reason you can't have loosely coupled "apps"
running under the same memory footprint sharing nothing more than a
common login.

I started to write more on the subject and it turned into a whole article:

http://darwinweb.net/article/Modular_Development_On_Rails

What do you think?

It is if that is your viewpoint. Fundamentally the object map is a
network, not a tree. So a directory hierarchy can only ever show one
viewpoint. And the current one is as good a compromise as any.

Perhaps a solution is a generator that can creates a different view
using symbolic links?

Either that or a semantic filesystem... :slight_smile:

NeilW

My experience with building an extension system for Radiant tells me
that it is. Radiant extensions provide an app directory with places
for controllers, views, and models--just like Rails--and the whole
thing seems extremely heavyweight.

Granted I can easily fix the layout for the Radiant extension system,
but that would kind of defeat the purpose. Radiant extensions are
supposed to look like mini-rails apps.

I believe the whole thing reveals a systemic problem with the way the
Rails directory structure was designed. A bold claim, I know, but I
believe it's true.

Again, what exactly is the point of organizing files by type? Our
applications have logical structure, why not reflect that in directory
layout?

Perhaps the example laid out in my article is a bit extreme. I'd see
it as a win if we just moved to a directory per related controller,
model, and views:

app/
  application/
    application_controller.rb
    application_layout.rhtml
  accounts/
    account.rb
    accounts_controller.rb
    accounts_helper.rb
    accounts_edit.rhtml
    accounts_login.rhtml
    accounts_signup.rhtml
  posts/
    post.rb
    posts_controller.rb
    posts_helper.rb
    posts_index.rhtml
    posts_archive.rhtml
    posts_post.rhtml
  comments/
    comment.rb
    comments_controller.rb
    comments_helper.rb
    comments_list.rhtml

Is that not 10 times cleaner and easier to navigate? Now I can `mate
app/comments` to work on a single controller and related views.

Please, set the issues of backwards compatibility aside or personal
preference and argue with merit alone.

Because of "configurable load paths, controller paths, and view
paths", I believe that what John Long is proposing is already
possible. It would just take some ingenuity and configuration effort.
So I really don't see the sense in lobbying for *everyone* using Rails
to change over to that alternate directory structure.

Cheers,
Obie

Perhaps the example laid out in my article is a bit extreme. I'd see
it as a win if we just moved to a directory per related controller,
model, and views:

[snipped example]

Is that not 10 times cleaner and easier to navigate? Now I can `mate
app/comments` to work on a single controller and related views.

Sure that's nice and easy to look at, but it rather assumes that you can take each model and assign it to a controller. The rails apps I work on would not fit so easily into that schema: many models have dependencies on other models, many controllers pull together several models. A lot of our models would have to either somehow live in 3 different places or I'd somehow have to arbitrarily pick one controller/component as the 'master' one for that model. Other models don't have corresponding controllers at all.

Fred

That's not necessarily true. For those models you could drop them in
the "app/application" folder with "application_controller.rb" or even
in a standard "shared" directory. There are any number of ways of
doing it and keeping it clean.

There is a definite benefit to being able to look at all the models or
controllers for an application together, or all the controllers. You
might convince me that your idea is BETTER, but you won't convince me
that it is universally better, or that organization by type is somehow
redundant and carries semantic benefit.

When you are working on radiant extensions of course it seems
heavyweight because you are talking about modular pieces of
functionality that are supposed to be tight and contained by design.

However Rails was designed to build monolithic applications, and when
you 10 heavily interdependent controllers backed by 20 models, you're
getting a good number of files per directory. Breaking them down into
modules may not be straightforward at all. If models are used evenly
by many controllers where do they go? Suddenly you find yourself
creating many modules just so the hierarchy is logical, and you end up
with the converse problem: that you have too many modules, none of
which are re-usable, when you could have everything in a single
controllers and models directory.

I totally understand how a Radiant extension just feels bloated with
all those directories, so I support some kind lighter-weight packaging
such as you propose. I think Obie is on the right track that we are
close to being able to do this already, but it will definitely need
some optimization as we can't just go filling up the view_paths and
controller paths willy-nilly everytime we add a module.

My experience with building an extension system for Radiant tells me
that it is. Radiant extensions provide an app directory with places
for controllers, views, and models--just like Rails--and the whole
thing seems extremely heavyweight.

The reality is that your use case is a tiny subset of the total number
of rails users. Most of us are trying to solve a specific problem,
rather than trying to glue together 'mini apps'. Rails is a web
development framework, not a chunk of glue for plugins.

Please, set the issues of backwards compatibility aside or personal
preference and argue with merit alone.

We have hundreds of thousands of downloads now. Setting aside
backwards compatibility isn't an option. As for merit, your proposal
only makes sense if we believe that reusable mini apps will ever be a
viable technique for the majority of developers. That's debatable at
best.

I can't see what is to stop you from releasing a plugin which uses our
current configuration options for controller model and view paths to
supports this functionality. Before we'd seriously consider
including this kind of functionality it would have to be *seriously*
battle tested, and considered an improvement by everyone that tried
it.

I definitely agree that a plugin would be the best way to test the
functionality before it could be seriously considered for core. I'm
just trying to gauge the value of the idea and the openness of the
core team before I attempt something that strenuous.

Although I don't see the practicality of grouping everything like you
intend, I could see grouping the controller helpers and views
together, but leaving the models separate:

app/
  application/
    application_controller.rb
    application_helper.rb
    views/
      layout.rhtml
  accounts/
    accounts_controller.rb
    accounts_helper.rb
    views/
      layout.rhtml
      edit.rhtml
      login.rhtml
      signup.rhtml
  models/
    account.rb

Typically the helper and views are related to the controller itself,
so this seems resonable.

nkryptic