Difference between revisions of "MySQL/Optimization"

From Freephile Wiki
Jump to navigation Jump to search
m (added Category:Database using HotCat)
(4 intermediate revisions by 2 users not shown)
Line 1: Line 1:
 
== Caching ==
 
== Caching ==
One thing you should look at is the query cache.  By default, the query cache is turned off.  You should turn it on, and set it in the tens of megabytes to see how it affects performance (like 80MB for starters).  A huge cache will hurt performance (don't give it 4GB just because you have 32GB of RAM).  But some amount of cache will definitely improve performance.  So you just have to find the sweetspot for your hardware and application.  That being said, if your tables are exclusively InnoDB  tables, then you might be able to turn query cache off. <ref>Hayden James: https://haydenjames.io/mysql-query-cache-size-performance/</ref> <ref>https://dev.mysql.com/doc/refman/5.7/en/query-cache-status-and-maintenance.html</ref> <ref>https://dba.stackexchange.com/questions/167271/mariadb-mysql-tuner-report-confusing</ref> <ref>https://dba.stackexchange.com/questions/66774/why-query-cache-type-is-disabled-by-default-start-from-mysql-5-6/66796#66796</ref>
+
One thing you should look at is the query cache.  By default, the query cache is turned off.  You should turn it on, and set it in the tens of megabytes to see how it affects performance (like 80MB for starters).  A huge cache will hurt performance (don't give it 4GB just because you have 32GB of RAM).  But some amount of cache will definitely improve performance.  So you just have to find the sweetspot for your hardware and application.  That being said, if your tables are exclusively InnoDB  tables, then you might be able to turn query cache off. <ref>Hayden James: https://haydenjames.io/mysql-query-cache-size-performance/</ref> <ref>https://dev.mysql.com/doc/refman/5.7/en/query-cache-status-and-maintenance.html</ref>
  
 
The cache is for SELECT queries obviously.  Any insert or update invalidates the cache so you can't get stale data from cache.  Low memory prunes indicates how many queries are being removed to make room for new queries that are being cached and the cache prune strategy is LRU (least recently used) so that queries that are seen the most frequently are the ones that get priority for caching.  It wouldn't make much sense to cache a query that is rare v. a query that is used on every page load.
 
The cache is for SELECT queries obviously.  Any insert or update invalidates the cache so you can't get stale data from cache.  Low memory prunes indicates how many queries are being removed to make room for new queries that are being cached and the cache prune strategy is LRU (least recently used) so that queries that are seen the most frequently are the ones that get priority for caching.  It wouldn't make much sense to cache a query that is rare v. a query that is used on every page load.
Line 28: Line 28:
 
WHERE VARIABLE_NAME LIKE 'QCACHE%'
 
WHERE VARIABLE_NAME LIKE 'QCACHE%'
 
) AS stats;
 
) AS stats;
</source>
 
To make your changes permanent, set it in your global option configuration file (/etc/mysql/my.cnf)
 
<source lang="ini">
 
query_cache_size = 80M
 
 
</source>
 
</source>
  
Line 37: Line 33:
 
Look for long-running transactions; and deadlocks
 
Look for long-running transactions; and deadlocks
  
The best tuning parameters are <ref>Bill Karwin, Percona http://www.slideshare.net/billkarwin/mysql-55-guide-to-innodb-status</ref>
+
The best tuning parameters are  
<ol>
+
# <code>innodb_buffer_pool_size</code> as much as you can spare after OS and MySQL
<li><code>innodb_buffer_pool_size</code> as much as you can spare after OS and MySQL
+
# <code>innodb_log_file_size</code> at least enough for 60 minutes of log writes (because sequential writes to a log file are faster than disk)
 
+
# <code>innodb_io_capacity</code> based on your disk IOPS
A rule of thumb would be to make it 10% larger than your database size.  If it's 10% smaller than your db, you probably won't notice a problem.  This is one reason to run your DB on a dedicated server: so that you can allocate as much memory as possible to the database without worrying about other OS and application memory requirements.<ref>https://www.percona.com/blog/2011/04/04/innodb-flushing-theory-and-solutions/</ref><ref>https://www.percona.com/blog/2007/11/03/choosing-innodb_buffer_pool_size/</ref>
+
<ref>Bill Karwin, Percona http://www.slideshare.net/billkarwin/mysql-55-guide-to-innodb-status</ref> <ref>https://www.percona.com/blog/2008/11/21/how-to-calculate-a-good-innodb-log-file-size/</ref> <ref>https://www.percona.com/blog/2011/04/04/innodb-flushing-theory-and-solutions/</ref><ref>https://www.percona.com/blog/2007/11/03/choosing-innodb_buffer_pool_size/</ref>
 
+
<source lang="bash">
See [https://dev.mysql.com/doc/refman/5.7/en/innodb-buffer-pool-resize.html the manual] for resizing the buffer.
+
SHOW ENGINE INNODB STATUS\G;
<li><code>innodb_log_file_size</code> at least enough for 60 minutes of log writes (because sequential writes to a log file are faster than disk) <ref>https://www.percona.com/blog/2008/11/21/how-to-calculate-a-good-innodb-log-file-size/</ref> Do this at peak usage, or use a bigger time window for sampling.
 
<source lang="mysql">
 
-- turn on the pager; setting it to 'grep sequence'
 
pager grep sequence
 
-- run our status report; wait 60 seconds and run it again
 
show engine innodb status\G select sleep(60); show engine innodb status\G
 
-- turn the pager off
 
nopager
 
</source>
 
Now you take the last number minus the first number, and convert to MB
 
<source lang="mysql">
 
select (43599802491 - 43599758916) / 1024 / 1024 as MB_per_minute;
 
 
</source>
 
</source>
Multiply that number by 60 to get an hours worth of log (and divide by 2 because there are two log files) Compare that to the current configuration
 
<source lang="mysql">
 
SHOW variables like 'innodb_log_file_size';
 
-- can be written as a SELECT statement using shorthand like this, and the value display is not limited
 
SELECT @@innodb_log_file_size;
 
</source>
 
The default is 5242880 which is 5MB
 
 
<li><code>innodb_io_capacity</code> based on your disk IOPS
 
</ol>
 
  
 
{{References}}
 
{{References}}
  
 
[[Category:Database]]
 
[[Category:Database]]
[[Category:MySQL]]
 

Revision as of 11:07, 4 August 2016

Caching[edit | edit source]

One thing you should look at is the query cache. By default, the query cache is turned off. You should turn it on, and set it in the tens of megabytes to see how it affects performance (like 80MB for starters). A huge cache will hurt performance (don't give it 4GB just because you have 32GB of RAM). But some amount of cache will definitely improve performance. So you just have to find the sweetspot for your hardware and application. That being said, if your tables are exclusively InnoDB tables, then you might be able to turn query cache off. [1] [2]

The cache is for SELECT queries obviously. Any insert or update invalidates the cache so you can't get stale data from cache. Low memory prunes indicates how many queries are being removed to make room for new queries that are being cached and the cache prune strategy is LRU (least recently used) so that queries that are seen the most frequently are the ones that get priority for caching. It wouldn't make much sense to cache a query that is rare v. a query that is used on every page load.

-- check the settings
SHOW VARIABLES LIKE '%cache%';
-- check the status
SHOW STATUS LIKE 'Qcache%';
-- turn it on 
SET GLOBAL query_cache_size = 80000000;
-- Measure performance
-- Uncomment line 2 to check the math
SELECT FORMAT(((QCACHE_HITS / (Qcache_hits + Qcache_inserts + Qcache_not_cached))*100),2) AS query_cache_hit_rate_percentage
--, QCACHE_FREE_BLOCKS, QCACHE_FREE_MEMORY, QCACHE_HITS, QCACHE_INSERTS, QCACHE_LOWMEM_PRUNES, QCACHE_NOT_CACHED, QCACHE_QUERIES_IN_CACHE, QCACHE_TOTAL_BLOCKS
FROM (
SELECT
MAX(IF(VARIABLE_NAME = 'QCACHE_FREE_BLOCKS', VARIABLE_VALUE, NULL)) QCACHE_FREE_BLOCKS,
MAX(IF(VARIABLE_NAME = 'QCACHE_FREE_MEMORY', VARIABLE_VALUE, NULL)) QCACHE_FREE_MEMORY,
MAX(IF(VARIABLE_NAME = 'QCACHE_HITS', VARIABLE_VALUE, NULL)) QCACHE_HITS,
MAX(IF(VARIABLE_NAME = 'QCACHE_INSERTS', VARIABLE_VALUE, NULL)) QCACHE_INSERTS,
MAX(IF(VARIABLE_NAME = 'QCACHE_LOWMEM_PRUNES', VARIABLE_VALUE, NULL)) QCACHE_LOWMEM_PRUNES,
MAX(IF(VARIABLE_NAME = 'QCACHE_NOT_CACHED', VARIABLE_VALUE, NULL)) QCACHE_NOT_CACHED,
MAX(IF(VARIABLE_NAME = 'QCACHE_QUERIES_IN_CACHE', VARIABLE_VALUE, NULL)) QCACHE_QUERIES_IN_CACHE,
MAX(IF(VARIABLE_NAME = 'QCACHE_TOTAL_BLOCKS', VARIABLE_VALUE, NULL)) QCACHE_TOTAL_BLOCKS
FROM information_schema.global_status
WHERE VARIABLE_NAME LIKE 'QCACHE%'
) AS stats;

InnoDB status[edit | edit source]

Look for long-running transactions; and deadlocks

The best tuning parameters are

  1. innodb_buffer_pool_size as much as you can spare after OS and MySQL
  2. innodb_log_file_size at least enough for 60 minutes of log writes (because sequential writes to a log file are faster than disk)
  3. innodb_io_capacity based on your disk IOPS

[3] [4] [5][6]

SHOW ENGINE INNODB STATUS\G;

References[edit source]