yahonda
(Yasuo Honda)
April 21, 2016, 7:57pm
1
Hi,
This is my first post to Rails core mailing list.
I’d like to propose that Rails 5 minimum version of SQLite 3.8 or higher.
( Here I am talking about SQLite database itself, not talking about sqlite3 gem. )
Usually I can open a pull request, but I need to know how to support SQLite 3.8 at Travis CI environment,
which can currently runs 3.7.11 by default or 3.7.15 which causes failures at ActiveRecord unit test.
Recently there are some issues reported. When more than 2 migrations migrated at the same time it fails with
ActiveRecord::StatementInvalid: SQLite3::SQLException: near ",": syntax error: INSERT INTO "schema_migrations" (version)
.
opened 01:09PM - 23 Mar 16 UTC
closed 02:00PM - 23 Mar 16 UTC
### Steps to reproduce
- Clone the repo and checkout "rails5" branch.
- Set up T… ravisCI.
- Change something small, such as add blank line to README.
- Push to GitHub and watch TravisCI.
### Expected behavior
Expected not to get ActiveRecord::StatementInvalid error.
This behavior did not occur on the master (Rails 4.2.6) branch.
### Actual behavior
I have been working on upgrading our local Ruby Users Group web
site from Rails 4.2 to Rails 5 in repo's "rails5" branch and
it fails TravisCI run (works fine locally) with this error message:
```
ActiveRecord::StatementInvalid: SQLite3::SQLException: near
",": syntax error: INSERT INTO "schema_migrations" (version)
VALUES ('20120321020431'), ('20120321184254')
```
- Code is at: https://github.com/atlrug/atlrug4/tree/rails5
- Last TravisCI build: https://travis-ci.org/atlrug/atlrug4/builds/117959091
### System configuration
**Rails version**: 5.0.0.beta3
**Ruby version**: 2.3.0
Thanks
opened 05:33PM - 21 Apr 16 UTC
closed 05:48PM - 21 Apr 16 UTC
activerecord
I ran into issues with migrations (either SQLite3 exception, see below, or an er… ror saying I have some pending migrations, but then it would throw an error that tables already exist...) and figured out that the problem appears whenever I have 3+ migrations. IIRC `db:reset` task is not the only one way for triggering this bug---I think ran into this also when redoing migrations (in development I often modify my migrations, remigrate them and rerun tests).
### Steps to reproduce
Minimal demo here: https://github.com/xHire/ror5-db_reset Simply run:
```
$ bin/rails db:migrate
$ bin/rails db:reset
```
### Expected behavior
Successful execution of the `db:reset` task.
### Actual behavior
```
$ bin/rails db:reset
Dropped database 'db/development.sqlite3'
Dropped database 'db/test.sqlite3'
Created database 'db/development.sqlite3'
Created database 'db/test.sqlite3'
-- create_table("authors", {:force=>:cascade})
-> 0.1721s
-- create_table("books", {:force=>:cascade})
-> 0.3224s
-- create_table("chapters", {:force=>:cascade})
-> 0.4233s
-- initialize_schema_migrations_table()
-> 0.1574s
rails aborted!
ActiveRecord::StatementInvalid: SQLite3::SQLException: near ",": syntax error: INSERT INTO "schema_migrations" (version) VALUES ('20160421154718'), ('20160421154709')
/home/xhire/.bundler/ruby/2.2.0/rails-81f4a7dce42e/activerecord/lib/active_record/connection_adapters/sqlite3_adapter.rb:225:in `block in execute'
/home/xhire/.bundler/ruby/2.2.0/rails-81f4a7dce42e/activerecord/lib/active_record/connection_adapters/abstract_adapter.rb:547:in `block in log'
/home/xhire/.bundler/ruby/2.2.0/rails-81f4a7dce42e/activesupport/lib/active_support/notifications/instrumenter.rb:21:in `instrument'
/home/xhire/.bundler/ruby/2.2.0/rails-81f4a7dce42e/activerecord/lib/active_record/connection_adapters/abstract_adapter.rb:541:in `log'
/home/xhire/.bundler/ruby/2.2.0/rails-81f4a7dce42e/activerecord/lib/active_record/connection_adapters/sqlite3_adapter.rb:225:in `execute'
/home/xhire/.bundler/ruby/2.2.0/rails-81f4a7dce42e/activerecord/lib/active_record/connection_adapters/abstract/schema_statements.rb:1029:in `assume_migrated_upto_version'
/home/xhire/.bundler/ruby/2.2.0/rails-81f4a7dce42e/activerecord/lib/active_record/schema.rb:52:in `define'
/home/xhire/.bundler/ruby/2.2.0/rails-81f4a7dce42e/activerecord/lib/active_record/schema.rb:44:in `define'
/home/xhire/tmp/ror5-db_failures/db/schema.rb:14:in `<top (required)>'
/home/xhire/.bundler/ruby/2.2.0/rails-81f4a7dce42e/activesupport/lib/active_support/dependencies.rb:287:in `load'
/home/xhire/.bundler/ruby/2.2.0/rails-81f4a7dce42e/activesupport/lib/active_support/dependencies.rb:287:in `block in load'
/home/xhire/.bundler/ruby/2.2.0/rails-81f4a7dce42e/activesupport/lib/active_support/dependencies.rb:259:in `load_dependency'
/home/xhire/.bundler/ruby/2.2.0/rails-81f4a7dce42e/activesupport/lib/active_support/dependencies.rb:287:in `load'
/home/xhire/.bundler/ruby/2.2.0/rails-81f4a7dce42e/activerecord/lib/active_record/tasks/database_tasks.rb:223:in `load_schema'
/home/xhire/.bundler/ruby/2.2.0/rails-81f4a7dce42e/activerecord/lib/active_record/tasks/database_tasks.rb:253:in `block in load_schema_current'
/home/xhire/.bundler/ruby/2.2.0/rails-81f4a7dce42e/activerecord/lib/active_record/tasks/database_tasks.rb:292:in `block in each_current_configuration'
/home/xhire/.bundler/ruby/2.2.0/rails-81f4a7dce42e/activerecord/lib/active_record/tasks/database_tasks.rb:291:in `each'
/home/xhire/.bundler/ruby/2.2.0/rails-81f4a7dce42e/activerecord/lib/active_record/tasks/database_tasks.rb:291:in `each_current_configuration'
/home/xhire/.bundler/ruby/2.2.0/rails-81f4a7dce42e/activerecord/lib/active_record/tasks/database_tasks.rb:252:in `load_schema_current'
/home/xhire/.bundler/ruby/2.2.0/rails-81f4a7dce42e/activerecord/lib/active_record/railties/databases.rake:260:in `block (3 levels) in <top (required)>'
/home/xhire/.bundler/ruby/2.2.0/rails-81f4a7dce42e/activerecord/lib/active_record/railties/databases.rake:264:in `block (3 levels) in <top (required)>'
/home/xhire/.bundler/ruby/2.2.0/rails-81f4a7dce42e/railties/lib/rails/commands/rake_proxy.rb:13:in `block in run_rake_task'
/home/xhire/.bundler/ruby/2.2.0/rails-81f4a7dce42e/railties/lib/rails/commands/rake_proxy.rb:10:in `run_rake_task'
/home/xhire/.bundler/ruby/2.2.0/rails-81f4a7dce42e/railties/lib/rails/commands/commands_tasks.rb:51:in `run_command!'
/home/xhire/.bundler/ruby/2.2.0/rails-81f4a7dce42e/railties/lib/rails/commands.rb:18:in `<top (required)>'
bin/rails:4:in `require'
bin/rails:4:in `<main>'
SQLite3::SQLException: near ",": syntax error
/home/xhire/.bundler/ruby/2.2.0/rails-81f4a7dce42e/activerecord/lib/active_record/connection_adapters/sqlite3_adapter.rb:225:in `block in execute'
/home/xhire/.bundler/ruby/2.2.0/rails-81f4a7dce42e/activerecord/lib/active_record/connection_adapters/abstract_adapter.rb:547:in `block in log'
/home/xhire/.bundler/ruby/2.2.0/rails-81f4a7dce42e/activesupport/lib/active_support/notifications/instrumenter.rb:21:in `instrument'
/home/xhire/.bundler/ruby/2.2.0/rails-81f4a7dce42e/activerecord/lib/active_record/connection_adapters/abstract_adapter.rb:541:in `log'
/home/xhire/.bundler/ruby/2.2.0/rails-81f4a7dce42e/activerecord/lib/active_record/connection_adapters/sqlite3_adapter.rb:225:in `execute'
/home/xhire/.bundler/ruby/2.2.0/rails-81f4a7dce42e/activerecord/lib/active_record/connection_adapters/abstract/schema_statements.rb:1029:in `assume_migrated_upto_version'
/home/xhire/.bundler/ruby/2.2.0/rails-81f4a7dce42e/activerecord/lib/active_record/schema.rb:52:in `define'
/home/xhire/.bundler/ruby/2.2.0/rails-81f4a7dce42e/activerecord/lib/active_record/schema.rb:44:in `define'
/home/xhire/tmp/ror5-db_failures/db/schema.rb:14:in `<top (required)>'
/home/xhire/.bundler/ruby/2.2.0/rails-81f4a7dce42e/activesupport/lib/active_support/dependencies.rb:287:in `load'
/home/xhire/.bundler/ruby/2.2.0/rails-81f4a7dce42e/activesupport/lib/active_support/dependencies.rb:287:in `block in load'
/home/xhire/.bundler/ruby/2.2.0/rails-81f4a7dce42e/activesupport/lib/active_support/dependencies.rb:259:in `load_dependency'
/home/xhire/.bundler/ruby/2.2.0/rails-81f4a7dce42e/activesupport/lib/active_support/dependencies.rb:287:in `load'
/home/xhire/.bundler/ruby/2.2.0/rails-81f4a7dce42e/activerecord/lib/active_record/tasks/database_tasks.rb:223:in `load_schema'
/home/xhire/.bundler/ruby/2.2.0/rails-81f4a7dce42e/activerecord/lib/active_record/tasks/database_tasks.rb:253:in `block in load_schema_current'
/home/xhire/.bundler/ruby/2.2.0/rails-81f4a7dce42e/activerecord/lib/active_record/tasks/database_tasks.rb:292:in `block in each_current_configuration'
/home/xhire/.bundler/ruby/2.2.0/rails-81f4a7dce42e/activerecord/lib/active_record/tasks/database_tasks.rb:291:in `each'
/home/xhire/.bundler/ruby/2.2.0/rails-81f4a7dce42e/activerecord/lib/active_record/tasks/database_tasks.rb:291:in `each_current_configuration'
/home/xhire/.bundler/ruby/2.2.0/rails-81f4a7dce42e/activerecord/lib/active_record/tasks/database_tasks.rb:252:in `load_schema_current'
/home/xhire/.bundler/ruby/2.2.0/rails-81f4a7dce42e/activerecord/lib/active_record/railties/databases.rake:260:in `block (3 levels) in <top (required)>'
/home/xhire/.bundler/ruby/2.2.0/rails-81f4a7dce42e/activerecord/lib/active_record/railties/databases.rake:264:in `block (3 levels) in <top (required)>'
/home/xhire/.bundler/ruby/2.2.0/rails-81f4a7dce42e/railties/lib/rails/commands/rake_proxy.rb:13:in `block in run_rake_task'
/home/xhire/.bundler/ruby/2.2.0/rails-81f4a7dce42e/railties/lib/rails/commands/rake_proxy.rb:10:in `run_rake_task'
/home/xhire/.bundler/ruby/2.2.0/rails-81f4a7dce42e/railties/lib/rails/commands/commands_tasks.rb:51:in `run_command!'
/home/xhire/.bundler/ruby/2.2.0/rails-81f4a7dce42e/railties/lib/rails/commands.rb:18:in `<top (required)>'
bin/rails:4:in `require'
bin/rails:4:in `<main>'
Tasks: TOP => db:schema:load
(See full trace by running task with --trace)
```
### System configuration
**Rails version**: beta3/edge (81f4a7d)
**Ruby version**: 2.2.3p173
I've tested it with Rails 4.2.6 too, but everything works fine there.
Since this commit will be available to Rails users who migrated to Rails 5. It has not backported to 4.2 or older version of Rails.
committed 09:17AM - 27 Jan 16 UTC
We found that inserting all 600 schema_migrations for our mid-sized app takes ab… out a minute on a cloud based CI environment.
I assume that the original code did not use multi-row-insert because SQLite3 was not supporting the syntax back then,
but it's been supported since 3.7.11: http://www.sqlite.org/releaselog/3_7_11.html
Ubuntu releases and SQLite versions
Refer Redirecting…
Thanks,
bitsweat
(Jeremy Daer)
April 22, 2016, 12:09am
2
Looks like OS X 10.9 Mavericks has SQLite 3.7.13, 10.10 Yosemite has 3.8.5, and 10.11 El Capitan has 3.8.10.2.
I think we’re fairly safe to bump the version requirement.
On the other hand, it’s pretty cheap/easy to support 3.7.x if we only do multi-insert if the db supports it:
if connection.supports_multi_insert?
…
Given that we’ve had multiple bug reports about it and how confusing it is to troubleshoot, that’s probably the friendlier choice.
Best,
Jeremy
yahonda
(Yasuo Honda)
April 22, 2016, 1:35pm
3
Thanks for the suggestion.
not bumping minimum requirement version to 3.8. The most biggest reason is still some active platform Ubuntu 12.04 LTS and OS X 10.9 Mavericks will be “unsupported” which installs SQLite 3.7.9 by default.
Here are supports_multi_insert?
return values for each adapters.
SQLite: true if sqlite_version >= ‘3.7.11’
https://www.sqlite.org/releaselog/3_7_11.html
Enhance the INSERT syntax to allow multiple rows to be inserted via the VALUES clause.
Rails 5 requires MySQL 5.0 or higher, which supports multi row insert.
Refer: Active Record supports MySQL >= 5.0 by kamipo · Pull Request #23458 · rails/rails · GitHub
https://dev.mysql.com/doc/refman/5.0/en/insert.html
“13.2.5 INSERT Syntax”
Rails 5 requires 9.1 or higher, which supports multi row insert.
Refer: The minimum supported version of PostgreSQL is now >= 9.1 by remomueller · Pull Request #23434 · rails/rails · GitHub
PostgreSQL 9.1 supports
I’m working on a pull request for this.
Thanks,