That is correct. Typically you would have one VirtualHost block:
<VirtualHost ...> ServerName xx.com ServerAlias www.xx.com ... </VirtualHost>
You would want to split that into two blocks:
<VirtualHost ...> ServerName xx.com </VirtualHost>
<VirtualHost ...> ServerName www.xx.com </VirtualHost>
