Tuesday, March 2, 2021

Invisible MySQL?

 Is MySQL going invisible?  Invisible Indexes were included in MySQL 8.0 and now with version 8.0.23 we have Invisible Columns.

Indexes You Can Not See!

The value of the invisible index is that it allows you to make an index disappear from the view of the optimizer.  In the distant days before 8.0, you would often delete an index you were pretty much definitively positive nobody or no query was using.  And then you would find out that yes, not only was that index you just deleted necessary to everyone in the galaxy (but maybe you)  but it was going to take some serious clock time to rebuild that index. 

But with Invisible Indexes, you issue a command like ALTER TABLE t1 ALTER INDEX i_idx INVISIBLE; and it was removed from use.  Now you can run EXPLAIN on your queries and compare results.  And if you want that index back among the visible, ALTER TABLE t1 ALTER INDEX i_idx VISIBLE;  returns you to full functionality.

I do recommend making an index invisible as part of a process of decommissioning an index, similar to a soft delete of a column to avoid hurried recreation.  And the Sys Schema will show you indexes that have not been used, just make sure you have long enough of a time period to let those queries that only run once a week/month/quarter or longer show themselves.  

Columns You Can Not See

MySQL 8.0.23 now allows you to have columns you can sort of not see.  There are not really invisible or obfuscated but those columns are harder to see.  If we create a table with an invisible column we have to explicitly call out that column to see the values as a * wildcard will not return the value.

SQL > create table stuff (id serial, c1 int, c2 int invisible);
Query OK, 0 rows affected (0.0393 sec)
SQL > insert into stuff (c1, c2) values row(1,2), row(3,4), row(5,6);
Query OK, 3 rows affected (0.0073 sec)
Records: 3  Duplicates: 0  Warnings: 0
SQL > select * from stuff;  <- the wildcard, no invisible column
+----+----+
| id | c1 |
+----+----+
|  1 |  1 |
|  2 |  3 |
|  3 |  5 |
+----+----+
3 rows in set (0.0005 sec)
SQL > select id,c1,c2 from stuff; <- have to call c2 to see c2
+----+----+----+
| id | c1 | c2 |
+----+----+----+
|  1 |  1 |  2 |
|  2 |  3 |  4 |
|  3 |  5 |  6 |
+----+----+----+
3 rows in set (0.0005 sec)

So you could somewhat hide a column by using this method but please do not call this secure.  This will allow you to add columns, say a primary key to a table lacking one, without having to worry about needing to modify existing queries.

And if you are creating a table from a table with an invisible column you need to explicitly reference it or you will not get that data.

Limitations?


The first limitation is that all columns can not invisible so at least one column needs to be visible.  Columns can be defined as NOT NULL and have DEFAULT values.  

What Else Is Going To Be Inviable?!


Well, in my job as a MySQL Community Manager, I do on rare occasion have access to software pre launch and I can tell you there are some really cool things in the product pipeline but I can not let you see them yet. (sorry, very bad joke)

Friday, February 26, 2021

Searching JSON Data

 A post on the MySQL forums caught my eye today. The distilled question was why was an index not being used to search JSON data.  The poster (in blue) had done most of the steps needed almost correctly but just missed the crucial steps. 

Suppose we have this table with an index on a virtual generated column as JSON attribute index pattern

```
CREATE TABLE applications (
id BINARY(16) PRIMARY KEY,
data JSON NOT NULL,
) ENGINE=InnoDB;
CREATE INDEX idx ON applications ((CAST(data->>"$.name" AS CHAR(10))));

Friday, February 5, 2021

MySQL & Friends FOSDEM Dev Room

 FOSDEM has become one of the biggest shows on the calendar of the MySQL Community Team.  FOSDEM itself, until this year, is barely contained chaos at a university in Brussels with no pre-enrollment for any of the dozens of themed rooms and no head count.  This year the event is virtual so you do not have to get to Belgium.


The MySQL and Friends Developer Room is Sunday and starts at 10:00am  Belgium time with a retrospective of the first twenty five years of MySQL. This is followed 15 other talks of about 25 minutes on everything from ARM to user management. 


TITLESPEAKERSTARTEND
25 years of MySQL – A RetrospectiveDave Stokes10:0010:20
Open Source Database Infrastructure with VitessShlomi Noach10:3010:55
From single MySQL instance to HA
The journey to InnoDB Cluster
Frédéric Descamps11:0011:25
Group Replication: Best Practices for Network DeployAníbal Pinto11:3011:55
Review of the Percona Operator from a DBA prospectiveMarco Tusa (the Grinch)12:0012:25
MySQL Router REST APIFrédéric Descamps12:3012:55

Better User Management under MySQL 8.0
Dave Stokes13:0013:25
Running MySQL on ARMKrunal Bauskar13:3013:55
Making MySQL-8.0 XA transaction processing crash safe
The key to use MySQL as storage nodes for distributed databases
Wei Zhao14:0014:25
Passing query attributes through the MySQL protocol
Extension to the wire format to allow data/metadata
Joro Kodinov14:3014:55
Linux /proc filesystem for MySQL DBAs
Sampling /proc content for troubleshooting
Valerii Kravchuk15:0015:20
Rewrite Your Complex MySQL Queries for Better PerformanceØystein Grøvlen15:3015:55
Automatic Asynchronous Replication Connection FailoverHemant Dangi16:0016:25
Sure you can run your database in kubernetes
Successfully run your MySQL NDB Cluster in kubernetes
Bernd Ocklin16:3016:55
18 Things To Do When You Have a MySQL BottleneckPeter Zaitsev17:0017:25
Hitchhiker’s Guide to MySQL
Don’t panic.
Liz van Dijk17:3017:55

Monday, January 25, 2021

Bye bye MySQL 5.6!

Adieu MySQL 5.6!

When you arrived in 2013 you had a lot of cool new features.  There was the NoSQL Memchache plug-in that was blazingly fast, an improved Performance Shema, full texted searching for InnoDB, big improvements in the optimizer, and great enhancements to replication. You were a great replacement for 5.5!

But in a few days you become history, a museum piece.  No more updates for big fixes for you.  You will become part of the 'back in the old MySQL 5.6' days stories senior DBAs and developers will tell.  You were a big improvement over 5.5 and a stepping stone to 5.7 & 8.0.  

You arrived with the mysql_config_editor, started the SHA256 password options, and you let us do some table alterations online.  

But now you time is passed and the MySQL Community Moves onward.  A lot of us old timers will hear your echoes in MySQL 8 and the newer users will not know what you brought to us.  

Bye bye 5.6. Rest well knowing you did your part.

JOINs - The Basics

 JOINs confuse a lot of those new to Structured Query Language (SQL). If you read the various web forums popularly used to ask questions then you know that using JOINs is pretty scary for a lot of folks. The sheer number of the 'how do I use JOIN to link two tables together?' questions is staggering. Not just because the JOIN itself is often obfuscated with relational algebra, Venn Diagrams, and the syntactical oddities of SQL -- and that is when you do find an online resource that tries to help. Plus adding to the frustration on top of all that are that the various options for using JSON can be downright perplexing.

Example 1

Let us start with an example of customers and their orders.  There is one table named customer for the information pertaining to the customer. There is another table with order information named orders holding the details of any orders from those folks in the customer table. 

Each of these tables has a column for the identification number of a customer.  In the customer table that column is named id and in the orders table that column is named customer_id.  Besides the inconsistencies in plurals of the table names, which is all too common in databases, there is the difference in column names for what is essentially the same data.  If you can get past that issue and realize that that you can use either column to link to the other table, then you are almost ready for the 'tricky stuff'.

To find the list of orders and their correspond customer, we can JOIN the two tables using the id/customer_id columns between the table with a simple JOIN. 

SELECT
orders.order_id
, customer.id
, customer.name  
from orders 
join customer 
on (customer.id = orders.customer_id) ;

And the results show the data from the orders table with the corresponding data with customer table.

The results of JOIN






LEFT Joins

But sometimes the two tables do not always evenly match up.  If we change the query above from join customer to LEFT join customer we get a much different result.

The results of LEFT JOIN




The LEFT JOIN displays all the orders and the corresponding customer information, if any.  If there is no corresponding data for the customer information, the server reports NULL(1).  In this case NULL is used to represent that we have no data for that element and order number 2 does not have a matching entry in the customer table.

RIGHT Joins

Changing the query to a RIGHT join will display all the customers and an orders they may have placed.

The Results of a RIGHT JOIN




The above shows that two orders have been placed by customers 1 & 2 and no orders from the customers 3 & 4.

CROSS Join

A CROSS Join will match every row in the first table with every row in the second table. Notice that the query is modified with no ON statement.  

The revised query for a
CROSSJOIN and the results











STRAIGHT_JOIN 

The STRAIGHT_JOIN is used to force the left table to be read first in cases where the optimizer wants to change things around for you.  With optimizer hints, this maybe redundant but it is still very handy.


The STRAIGHT_JOIN









The Natural JOIN

The NATURAL join creates an implicit join clause on the common columns in the two tables being joined. Common columns are columns that have the same name in both tables which means that you need to be VERY certain the columns with the same names are actually using the same data -- You do not want the 'id' for customers to get confused with those from the orders, employees, or other tables.


The Natural Join










(1) NULL

Null is used to indicate the absence of data. If you go back to the second example. the LEFT JOIN, we received records where there was no matching customer id number or name.  This could be caused by bad data in the tables, a badly entered record, or many other (frustrating) causes.  But the records that are holding NULLs can be found.  Hopefully I can loop back to that at a later date.



Tuesday, January 5, 2021

MySQL 5.6 End of Life in ONE MONTH!!!

 For the past year or so I have been reminding folks that MySQL 5.6 reaches End of Life Status in ONE MONTH!!  No more updates or security fixes!  Nostalgia is a fine things and I like antiques but not for my database!

So if you are on 5.6 please upgrade to 5.7 -- https://dev.mysql.com/doc/refman/5.7/en/upgrading.html and consider going to 8.0 - https://dev.mysql.com/doc/refman/8.0/en/upgrading.html  plus there is a5.7 to 8.0 upgrade checker in the new shell - https://dev.mysql.com/doc/mysql-shell/8.0/en/mysql-shell-utilities-upgrade.html

Friday, December 18, 2020

Holidays Gifts For Your MySQL DBA - 2020 Edition

 Dear Family, Friend, Co-Worker, or Acutance of a MySQL DBA,

You MySQL DBA knows that shopping for them this time of year is very hard as all they really need is a new instance of MySQL to make them ecstatic.   However they have downloaded this list of gifts, have printed this out, and circled some gift ideas from this blog -- to make it easier for you. 


1. Singing Tesla Coil Music Kit Plasma Loudspeaker Wireless Transmission Experiment Desktop 


Link

Musical Tesla Coil (also known as "Singing Tesla Coil"), sometimes called a zeusaphone, thoramin or musical lightning, is a form of plasma speaker. The music Tesla coil, It is a variety of a solid state Tesla coil that has been modified to produce musical tones by modulating its spark output. The resulting pitch is a low fidelity square wave like sound reminiscent of an analog synthesizer. The high-frequency signal acts in effect as a carrier wave; its frequency is significantly above human-audible sound frequencies, so that digital modulation can reproduce a recognizable pitch. The musical tone results directly from the passage of the spark through the air.

2. A new Cup


Link

Yes, a new mug is needed as we approach 2021!




3. Alternative Cup


Link

Maybe two new mugs! This time with a possibly naughty message!





4. Star Wars Death Star Popcorn Maker - Hot Air Style with Removable Bowl


HEALTHIER CHOICE - Uses hot-air popping method so no oil required. It makes popcorn a perfect snack for the whole family.

EASY OPERATION - Add kernels into the main chamber using the provided measuring cup, turn on the unit and watch the magic happen through the transparent cover!

HIGH EFFICIENCY - Pops evenly and quickly, with up to a 98% popping rate. Unique channel ensures constant hot air to pop about as quickly as a microwave (2-3 minutes).


5. A New Light Switch Cover


Link 

Change up that boring switch plate with something a little quirky!





6. A new blanket or pillow


Link 


Yes, that MySQL DBA needs at least one of these too!




7. New shirt


Link 

With the closing of many in person conferences, your DBA may be lacking in t-shirts so get them this one!