I'm somewhat of a newbie to agile practices, and I know the Rails community in general is pretty "up to speed" on agile methodologies, but if there's a better place for me to ask this question, let me know, and forgive the off topic post.
I'm trying to "grok" agile practices in "the real world". I've read the Agile Manifesto (http://agilemanifesto.org/), and I find it somewhat vague. Most of this is common sense, but the one thing that irks me (probably because I don't understand it fully) is that "responding to change" is more important than "following a plan".
Now, what I'm trying to get, is exactly HOW - in terms of technology and practices - we, as software developers, can respond to change *over* following a plan. Let me illustrate with a real life example.
I'm currently working a project for a cell phone company where the company supplies a list of employees and a list of sales, and what employee made which sales, and which of 4 different types of employees they are (retail sales, territory manager, store manager, business sales person, etc.). The goal of the project is to calculate the commission to be paid to each user (somewhere in the 200 users range - it's a small company).
One problem is that each of those four types has anywhere from 4 to 8 "sub parts" of their commissions. There's a separate total for activations, a separate total for renewals, accessory sales, and other types of sales, depending on the role. Each of these is calculated in an entirely different manner, and compared to a "floating modifier" table, that is different for each role type, for each type of calculation, then re-calculated, etc. It's a highly convoluted and screwed up process, but you know how it is with "management" in a corporation...
Now, I'm tying this in with a Ruby on Rails application that serves as the company intranet web portal for our entire company as well, so this is all tied to the User model within the application.
Here's where agile practices come into play, and where the "rub" is: I had an entire system finished and ready to go. Then management stepped in, and for the third time in two months, changed it all. Four user types got expanded to 9, and instead of 4-8 sub parts for each user calculated differently, that got expanded from 4-15. The changes were so sweeping that there was no way to simply "modify" existing code. I had to start over again from scratch.
Because it had to be tied to an already existent User model, I used some mixins to structure the different versions of the system (which I'm looking at making an entirely separate application, by the way). I'd consider this a somewhat "agile" method - keeping code modular; but what else could I do?
I also practice TDD fairly well (though I need to teach myself RSpec/ Cucumber etc.). So of course, each method had tests to back it up, and I put a LOT of time and effort into simplifying and normalizing the database to keep things within best practices, not duplicate data, and "future proof" as best as I could. I find TDD a *big* help with stuff like this, to "prove" that your code works in case anyone starts a fight or a political struggle (which is common where I work, unfortunately), but more importantly, to facilitate change by making sure that changes don't break existing functionality.
My problem is that short of : - TDD - Really thinking hard about how to structure your data - Using source control (I'm still with Subversion though I want to play with git a bit more, get a better feel for the differences between them) - and "modularizing" code conceptually (MVC, mixins, etc.)
I don't know what ELSE I can do, in terms of agile PRACTICES, to respond to change better.
In my experience, change is a daily to hourly occurrence. It's ridiculous in some cases, and I'm wondering: how far can we take the "agile" concept? How do we EXECUTE that concept on a day-to-day basis? What best practices are there for making change this frequent, this sweeping, more manageable?
I want to be a much better developer than I am, and be able to respond to these changes in a more "agile" way. I get stuff like this thrown at me all the time, and I'm sure I'm not alone in that, but I feel I may be behind the times in learning to respond to this better with better practices, development techniques, and possibly tools.
Can anybody point me to some resources, comment on how I can better respond to changes LIKE this (the above is only for illustrative purposes, I'm not looking for "answers" to this situation, just some better ways to approach anything similar to it now and in the future), etc.?
Thanks a lot for any input you can provide. And again, if this post is better suited to another Google talk group or elsewhere, let me know and I'll go put it there, and please pardon the intrusion.