Difference between revisions of "MySQL"

From Freephile Wiki
Jump to navigation Jump to search
(Add info about switching between pluggable auth)
(→‎Tools: steps to use Percona Toolkit)
 
(9 intermediate revisions by the same user not shown)
Line 1: Line 1:
=== Installation ===
+
===Installation===
 
To get started, [https://access.redhat.com/site/documentation/en-US/Red_Hat_Enterprise_Linux/6/html/Managing_Confined_Services/chap-Managing_Confined_Services-MySQL.html installing MySQL on RHEL] is as simple as <source lang="bash">yum install mysql-server</source>.  However, you might get a message that your system is not registered to Red Hat Subscription Management (RHN in the old days), and thus the packages are not visible.  To remedy this, you would of course use <code>subscription-manager</code> to register the host with Red Hat Subscription Management.
 
To get started, [https://access.redhat.com/site/documentation/en-US/Red_Hat_Enterprise_Linux/6/html/Managing_Confined_Services/chap-Managing_Confined_Services-MySQL.html installing MySQL on RHEL] is as simple as <source lang="bash">yum install mysql-server</source>.  However, you might get a message that your system is not registered to Red Hat Subscription Management (RHN in the old days), and thus the packages are not visible.  To remedy this, you would of course use <code>subscription-manager</code> to register the host with Red Hat Subscription Management.
  
Line 24: Line 24:
 
</source>
 
</source>
  
== Create a DB and user ==
+
==Create a DB and user==
 
<source lang="sql">
 
<source lang="sql">
 
CREATE DATABASE mediawiki;
 
CREATE DATABASE mediawiki;
GRANT ALL PRIVILEGES ON mediawiki.* TO 'example_user'@'localhost' IDENTIFIED BY 'secret';
+
GRANT ALL PRIVILEGES ON mediawiki.* TO 'example_user'@'localhost' IDENTIFIED BY 'secretpass';
 
</source>
 
</source>
  
Note: Since the introduction of [https://mariadb.com/kb/en/library/development-pluggable-authentication/ Pluggable Authentication] in MySQL 5.7 and also MariaDB since 5.2, there is an "[https://mariadb.com/kb/en/library/authentication-plugin-unix-socket/ unix_socket]" plugin which authenticates the current user by their login to the OS, and thus NOT a password. If you have a database with no user password, then you'll probably want to change it back to regular password auth <ref>https://www.percona.com/blog/2016/03/16/change-user-password-in-mysql-5-7-with-plugin-auth_socket/</ref>
+
==Set Password for Root user==
 +
The following command issues a new password for the root user while at the same time dictating which authentication plugin <ref>https://mariadb.com/kb/en/library/authentication-plugin-pam/</ref> to use.
 +
<source lang="sql">
 +
ALTER USER 'root'@'localhost' IDENTIFIED WITH mysql_native_password BY 'secretpass';
 +
</source>
 +
 
 +
 
 +
Note: Since the introduction of [https://mariadb.com/kb/en/library/development-pluggable-authentication/ Pluggable Authentication] in MySQL 5.7 and also MariaDB since 5.2, there is a "[https://mariadb.com/kb/en/library/authentication-plugin-unix-socket/ unix_socket]" plugin which authenticates the current user by their login to the OS, and thus NOT a password. If you have trouble setting the password, it's probably because you need to also specify the '''mysql_native_password''' auth at the same time <ref>https://www.percona.com/blog/2016/03/16/change-user-password-in-mysql-5-7-with-plugin-auth_socket/</ref>
  
== What's going on? ==
+
==What's going on?==
 
Want to find out what's going on in your MySQL server, but don't want to install client tools and access?  Just make sure that the General Query Log is turned on, and tail the log file:
 
Want to find out what's going on in your MySQL server, but don't want to install client tools and access?  Just make sure that the General Query Log is turned on, and tail the log file:
 
<source lang="sql">
 
<source lang="sql">
Line 55: Line 62:
 
If you want more info than you can get from the direct query logging, then try [http://mtop.sourceforge.net/|mtop]
 
If you want more info than you can get from the direct query logging, then try [http://mtop.sourceforge.net/|mtop]
  
== Show grants ==
+
==Show grants==
 
<code>SHOW GRANTS</code> only shows privs for the current user.
 
<code>SHOW GRANTS</code> only shows privs for the current user.
 
<source lang="sql">
 
<source lang="sql">
Line 67: Line 74:
 
</source>
 
</source>
  
== MariaDB Differences between Ubuntu and Debian ==
+
==Fixing CHARSET and COLLATION==
https://mariadb.com/kb/en/library/differences-in-mariadb-in-debian-and-ubuntu/
+
See the article about [[Collation]]
 +
==Working at the Console==
 +
Reading output from the mysql command line client is notoriously ugly/hard.  With the <code>-s --silent</code> or <code>-B --batch</code> options, you can get output that is more readable.
 +
 
 +
==Experts==
 +
Rick James https://mysql.rjweb.org/
 +
 
 +
Major Hayden https://major.io/tags/database/ https://github.com/major
 +
 
 +
==Tools==
 +
Aside from [[mysqldump]], there are also mysqlcheck,  mysql_upgrade and other [https://dev.mysql.com/doc/refman/5.7/en/programs-client.html client programs].
 +
 
 +
#[https://www.mysql.com/products/workbench/ MySQL Workbench] is a great visual tool, but behind several obstacles and Oracles.
 +
#[https://dbeaver.io/ DBeaver]
 +
#[https://github.com/major/MySQLTuner-perl/blob/master/INTERNALS.md MySQLTuner] by the aforementioned Major Hayden (he made a Python version too)
 +
#[https://www.percona.com/software/database-tools/percona-toolkit Percona Toolkit] <code>wget <nowiki>https://downloads.percona.com/downloads/percona-toolkit/3.5.7/binary/redhat/8/x86_64/percona-toolkit-3.5.7-1.el8.x86_64.rpm</nowiki></code>  <code>dnf install percona-toolkit-3.5.7-1.el8.x86_64.rpm</code>  <code>pt-mysql-summary</code>
 +
#http://mysqlsandbox.net/index.html MySQL sandbox could be useful for playing around with multiple databases in development.
 +
 
 +
 
 +
==Logs==
 +
Your data directory can fill up with binary logs unless you rotate them.
 +
 
 +
Information below comes from https://dba.stackexchange.com/questions/41050/is-it-safe-to-delete-mysql-bin-files
 +
 
 +
'''Don't''' just delete the files.  Use MySQL to do it.
 +
 
 +
Either pass the name of a particular file, or a timestamp and MySQL will clear out the binary logs up to the one just specified (or the date)
  
If you're collation and character sets are all latin1_swedish, but you want them to be UTF-8, then you can set it in the configuration file, and restart the database.
+
;Example :
 
<source lang="mysql">
 
<source lang="mysql">
show GLOBAL VARIABLES LIKE 'character_set_%';
+
-- consolidate the logs up to midnight 3 days ago
show VARIABLES LIKE 'collation%';
+
PURGE BINARY LOGS BEFORE DATE(NOW() - INTERVAL 3 DAY) + INTERVAL 0 SECOND;
 +
-- consolidate the logs up to the file specified
 +
PURGE BINARY LOGS TO `mysql-bin.000223`;
 
</source>
 
</source>
<pre>
 
# Variable_name, Value
 
'collation_connection', 'utf8_general_ci'
 
'collation_database', 'latin1_swedish_ci'
 
'collation_server', 'latin1_swedish_ci'
 
</pre>
 
  
This is the mess that I wound up with somehow:
+
 
 +
;Which File to choose?:
 +
 
 +
If you want to know which log files are in use so that you can remove superfluous ones without interrupting replication, go to your slave database and issue <tt>SHOW SLAVE STATUS\G
 +
 
 +
</tt>
 +
You'll see entries for
 +
 
 +
*Master_Log_File
 +
*Relay_Master_Log_File
 +
 
 +
When replication has little or no lag these are usually the same value. If there is a lot of replication lag, they will be different values.  Choose the value for <tt>Relay_Master_Log_File</tt> just to be safe.  Go back to the DB master and issue <tt>PURGE BINARY LOGS TO mysql-bin.000337</tt> where mysql-bin.000337 is the value of <tt>Relay_Master_Log_File</tt>
 +
 
 +
 
 +
;Persistence :
 +
 
 +
To persist this setup so your data directory doesn't fill up again, issue
 
<source lang="mysql">
 
<source lang="mysql">
SET @DATABASE_NAME = 'wiki_freephile';
+
SET GLOBAL expire_logs_days = 3;
SELECT
 
    TABLE_SCHEMA,
 
    COUNT(COLLATION_NAME) AS 'count',
 
    COLLATION_NAME
 
FROM
 
    information_schema.columns AS cols
 
WHERE
 
    table_schema = @DATABASE_NAME
 
GROUP BY COLLATION_NAME;
 
 
</source>
 
</source>
<pre>
+
To put the setting into effect immediately. And for the next time MySQL is restarted, add to <tt>/etc/my.cnf</tt>
wiki_freephile, 0,
+
<source lang="ini">
wiki_freephile, 11, binary
+
[mysqld]
wiki_freephile, 73, latin1_bin
+
 
wiki_freephile, 69, latin1_swedish_ci
+
expire_logs_days=3
wiki_freephile, 1, utf8_general_ci
 
</pre>
 
So, I fixed the defaults for each database:
 
<source lang="mysql">
 
ALTER DATABASE wiki_freephile CHARACTER SET = 'binary' COLLATE = 'BINARY';
 
ALTER DATABASE wiki_meta CHARACTER SET = 'binary' COLLATE = 'BINARY';
 
ALTER DATABASE wiki_wiki CHARACTER SET = 'binary' COLLATE = 'BINARY';
 
 
</source>
 
</source>
  
You can check the status of each table too:
+
Since the binlongs are used to recover the database state after a crash, this assumes that you are making database backups every day.  If you only backup once per week, you would want at least 7 days worth of binlogs.
<source lang="mysql">
 
SET @DATABASE_NAME = 'wiki_freephile';
 
SELECT TABLE_SCHEMA, TABLE_NAME, TABLE_COLLATION
 
FROM    information_schema.tables
 
WHERE  TABLE_SCHEMA = @DATABASE_NAME
 
AND TABLE_COLLATION not in ('binary');
 
</source>
 
  
To set the mysql server to utf8 <ref>https://mariadb.com/kb/en/library/setting-character-sets-and-collations/ https://scottlinux.com/2017/03/04/mysql-mariadb-set-character-set-and-collation-to-utf8/</ref>, add the following to <code>/etc/mysql/my.cnf</code> (Debian/Ubuntu) or <code>/etc/my.cnf</code> (Fedora/Centos/RHEL) under the '''<nowiki>[mysqld]</nowiki>''' section:
+
==Tuning Reference==
<pre>
 
character-set-server = utf8
 
collation-server = utf8_general_ci
 
skip-character-set-client-handshake
 
</pre>
 
Then restart mysql/mariadb:
 
*<code>service mariadb restart</code> for Debian/Ubuntu;
 
*<code>/bin/systemctl restart mariadb.service</code> for Fedora/Centos/RHEL
 
  
== Working at the Console ==
+
#https://mariadb.com/kb/en/library/configuring-mariadb-for-optimal-performance
Reading output from the mysql command line client is notoriously ugly/hard. With the <code>-s --silent</code> or <code>-B --batch</code> options, you can get output that is more readable.
+
#https://www.experts-exchange.com/questions/29060302/Tune-MySQL-to-32GB-RAM.html
 +
#https://www.tecmint.com/mysql-mariadb-performance-tuning-and-optimization/
 +
#https://www.percona.com/blog/10-mysql-performance-tuning-settings-after-installation/
 +
#https://www.percona.com/blog/2016/10/12/mysql-5-7-performance-tuning-immediately-after-installation/
 +
#https://www.percona.com/blog/2009/11/16/table_cache-negative-scalability/
  
 +
==MediaWiki==
 +
As of 2024-04-23 MediaWiki only requires MariaDB 10.3.x  or MySQL 5.7.x for installation<ref>https://www.mediawiki.org/wiki/Manual:Installation_requirements#Database_server</ref> However, these are out of support. MariaDB is available in versions ranging from [https://mariadb.com/kb/en/changes-and-improvements-in-mariadb-10-4/ 10.4] - 10.11 and 11.0 - 11.5 The latest release in the 10.4 series is 10.4.33, released on 2024-02-07
  
== Tools ==
+
Practically, this means that even if the database doesn't show errors, it could become vulnerable to attack or corruption at any time. And, tools like MySQLTuner, VSCode, PHPMyAdmin, etc. may not support the old version any longer.
Aside from [[mysqldump]], there are also mysqlcheck, mysql_upgrade and other [https://dev.mysql.com/doc/refman/5.7/en/programs-client.html client programs].
 
  
# MySQL Workbench is a great visual tool.
+
Starting in REL1_35, MediaWiki uses the [https://www.doctrine-project.org/projects/dbal.html Doctrine Database Abstraction Layer] (DBAL)
# [https://github.com/major/MySQLTuner-perl/blob/master/INTERNALS.md MySQLTuner]
 
# http://mysqlsandbox.net/index.html MySQL sandbox could be useful for playing around with multiple databases in development.
 
  
== Tuning Reference ==
+
<br />
# https://mariadb.com/kb/en/library/configuring-mariadb-for-optimal-performance
 
# https://www.experts-exchange.com/questions/29060302/Tune-MySQL-to-32GB-RAM.html
 
# https://www.tecmint.com/mysql-mariadb-performance-tuning-and-optimization/
 
# https://www.percona.com/blog/2014/01/28/10-mysql-performance-tuning-settings-after-installation/
 
# https://www.percona.com/blog/2016/10/12/mysql-5-7-performance-tuning-immediately-after-installation/
 
# https://www.percona.com/blog/2009/11/16/table_cache-negative-scalability/
 
  
== Footnotes ==
+
==Footnotes==
<References />
+
<references />
  
 
[[Category:Database]]
 
[[Category:Database]]

Latest revision as of 16:00, 24 April 2024

Installation[edit | edit source]

To get started, installing MySQL on RHEL is as simple as

yum install mysql-server

. However, you might get a message that your system is not registered to Red Hat Subscription Management (RHN in the old days), and thus the packages are not visible. To remedy this, you would of course use subscription-manager to register the host with Red Hat Subscription Management.

/usr/bin/mysqladmin -u root password 'new-password'
/usr/bin/mysqladmin -u root -h `hostname` password 'new-password'
# but it's easier and better to run
/usr/bin/mysql_secure_installation
# which will give you the option of removing the test database and removing the anonymous user created by default.  This is a must for production servers.

You can start the MySQL daemon with:

cd /usr ; /usr/bin/mysqld_safe &

You can test the MySQL daemon with mysql-test-run.pl

cd /usr/mysql-test ; perl mysql-test-run.pl

It's a good idea to setup a local configuration file for the system 'root' user, who can connect to the database server as db user 'root' by default when using the mysql command line client (ie. when you are user 'root' and type 'mysql' at the command line in a bash shell)

cat /root/.my.cnf
[client]
user=root
password=giveMeSomeDBgoodne55

Create a DB and user[edit | edit source]

CREATE DATABASE mediawiki;
GRANT ALL PRIVILEGES ON mediawiki.* TO 'example_user'@'localhost' IDENTIFIED BY 'secretpass';

Set Password for Root user[edit | edit source]

The following command issues a new password for the root user while at the same time dictating which authentication plugin [1] to use.

ALTER USER 'root'@'localhost' IDENTIFIED WITH mysql_native_password BY 'secretpass';


Note: Since the introduction of Pluggable Authentication in MySQL 5.7 and also MariaDB since 5.2, there is a "unix_socket" plugin which authenticates the current user by their login to the OS, and thus NOT a password. If you have trouble setting the password, it's probably because you need to also specify the mysql_native_password auth at the same time [2]

What's going on?[edit | edit source]

Want to find out what's going on in your MySQL server, but don't want to install client tools and access? Just make sure that the General Query Log is turned on, and tail the log file:

SHOW VARIABLES LIKE "general_log%";

+------------------+----------------------------+
| Variable_name    | Value                      |
+------------------+----------------------------+
| general_log      | OFF                        |
| general_log_file | /var/run/mysqld/mysqld.log |
+------------------+----------------------------+

mysql> SET GLOBAL general_log = 'ON';

Then you can tail the log file to see what it's doing:

tail -f /var/run/mysqld/mysqld.log

Be sure to turn off the log or your disk will fill up and you will not be happy.

mysql> SET GLOBAL general_log = 'OFF';

If you want more info than you can get from the direct query logging, then try [1]

Show grants[edit | edit source]

SHOW GRANTS only shows privs for the current user.

show grants;
show grants for 'root'@'localhost';

But this bash one-liner will show you all grants. -B --batch disables interactive behavior and doesn't use the history file -N --skip-column-names -e --execute executes a command and quits (disables --force and history file.)

mysql -B -N -e "SELECT DISTINCT CONCAT('SHOW GRANTS FOR ''',user,'''@''',host,''';') AS query FROM user" mysql | mysql

Fixing CHARSET and COLLATION[edit | edit source]

See the article about Collation

Working at the Console[edit | edit source]

Reading output from the mysql command line client is notoriously ugly/hard. With the -s --silent or -B --batch options, you can get output that is more readable.

Experts[edit | edit source]

Rick James https://mysql.rjweb.org/

Major Hayden https://major.io/tags/database/ https://github.com/major

Tools[edit | edit source]

Aside from mysqldump, there are also mysqlcheck, mysql_upgrade and other client programs.

  1. MySQL Workbench is a great visual tool, but behind several obstacles and Oracles.
  2. DBeaver
  3. MySQLTuner by the aforementioned Major Hayden (he made a Python version too)
  4. Percona Toolkit wget https://downloads.percona.com/downloads/percona-toolkit/3.5.7/binary/redhat/8/x86_64/percona-toolkit-3.5.7-1.el8.x86_64.rpm dnf install percona-toolkit-3.5.7-1.el8.x86_64.rpm pt-mysql-summary
  5. http://mysqlsandbox.net/index.html MySQL sandbox could be useful for playing around with multiple databases in development.


Logs[edit | edit source]

Your data directory can fill up with binary logs unless you rotate them.

Information below comes from https://dba.stackexchange.com/questions/41050/is-it-safe-to-delete-mysql-bin-files

Don't just delete the files. Use MySQL to do it.

Either pass the name of a particular file, or a timestamp and MySQL will clear out the binary logs up to the one just specified (or the date)

Example 
-- consolidate the logs up to midnight 3 days ago
PURGE BINARY LOGS BEFORE DATE(NOW() - INTERVAL 3 DAY) + INTERVAL 0 SECOND;
-- consolidate the logs up to the file specified
PURGE BINARY LOGS TO `mysql-bin.000223`;


Which File to choose?

If you want to know which log files are in use so that you can remove superfluous ones without interrupting replication, go to your slave database and issue SHOW SLAVE STATUS\G

You'll see entries for

  • Master_Log_File
  • Relay_Master_Log_File

When replication has little or no lag these are usually the same value. If there is a lot of replication lag, they will be different values. Choose the value for Relay_Master_Log_File just to be safe. Go back to the DB master and issue PURGE BINARY LOGS TO mysql-bin.000337 where mysql-bin.000337 is the value of Relay_Master_Log_File


Persistence 

To persist this setup so your data directory doesn't fill up again, issue

SET GLOBAL expire_logs_days = 3;

To put the setting into effect immediately. And for the next time MySQL is restarted, add to /etc/my.cnf

[mysqld] 

expire_logs_days=3

Since the binlongs are used to recover the database state after a crash, this assumes that you are making database backups every day. If you only backup once per week, you would want at least 7 days worth of binlogs.

Tuning Reference[edit | edit source]

  1. https://mariadb.com/kb/en/library/configuring-mariadb-for-optimal-performance
  2. https://www.experts-exchange.com/questions/29060302/Tune-MySQL-to-32GB-RAM.html
  3. https://www.tecmint.com/mysql-mariadb-performance-tuning-and-optimization/
  4. https://www.percona.com/blog/10-mysql-performance-tuning-settings-after-installation/
  5. https://www.percona.com/blog/2016/10/12/mysql-5-7-performance-tuning-immediately-after-installation/
  6. https://www.percona.com/blog/2009/11/16/table_cache-negative-scalability/

MediaWiki[edit | edit source]

As of 2024-04-23 MediaWiki only requires MariaDB 10.3.x or MySQL 5.7.x for installation[3] However, these are out of support. MariaDB is available in versions ranging from 10.4 - 10.11 and 11.0 - 11.5 The latest release in the 10.4 series is 10.4.33, released on 2024-02-07

Practically, this means that even if the database doesn't show errors, it could become vulnerable to attack or corruption at any time. And, tools like MySQLTuner, VSCode, PHPMyAdmin, etc. may not support the old version any longer.

Starting in REL1_35, MediaWiki uses the Doctrine Database Abstraction Layer (DBAL)


Footnotes[edit | edit source]