Blog

Does MySQL care about Windows users?

Well .. management does!

Sun/MySQL executives have at many occasions made it clear that Windows is very important for them.
Jonathan Schwartz: http://blogs.sun.com/jonathan/entry/winds_of_change_are_blowing
Robin Schumacher: http://dev.mysql.com/tech-resources/articles/mysql_on_windows.html

This is a very clear signal from management! Windows is an important platform and it is important for MySQL too. And you will find similar statements from key executives of MySQL. But do MySQL developers care about the signals from their management? Not much! Management is farther away than a small town in Russia (Danish slang for ’something we need not care about’) and Windows users are at a small village at some even more remote place!

Symptoms

1)
Examples of carelessness to users’ data:
http://bugs.mysql.com/bug.php?id=36493
http://bugs.mysql.com/bug.php?id=46258

These are the bugs that reflect carelessness in developing Windows specific software components. It is not a small detail – it is very fundamental for Windows. Configuration data etc.  are the property of the user – not the property of the MySQL installer!

2)
Issues with installers for Windows and the Configuration Wizard. Let us list a few (but there are dozens – search yourself):
http://bugs.mysql.com/bug.php?id=42820
http://bugs.mysql.com/bug.php?id=44073
http://bugs.mysql.com/bug.php?id=44073

Reality is that due to this, MySQL software is close to useless for lot of Windows users – should they be hit  by any of those issues – unless they get help to understand what is happening  (what they won’t get from MySQL – but many a times from us through our forums and support ticket system). Users are even scared when the Configuration Wizard returns the message ‘failed to apply security settings’.

All this started more than 2 years ago. Very little activity since then. It seems that MySQL developers think that this is too trivial to spend time on and that those people bothering about reporting such issues must be a bunch of teenage amateurs. They may be (some are!). But that does not a excuse not taking concerns seriously. It is true that all the issues are not very serious if you understand what is going on. But those users that don’t, hurry to uninstall everything related to MySQL (’failed .. security’ – Windows users are scared about that). Actually it is also a problem for us as a 3rd party vendor. People claim occasionally to our support that “SQLyog is insecure because I can connect to MySQL without using the password I have just defined when installing the server”. Every time we have to explain users that it is the Configuration Wizard that is insecure, and that with SQLyog (or any client) it is simple to do what the Configuration Wizard failed to do.

3)
‘lower_case_table_names’ server variable is seriously broken since MySQL 4.0 and it gets worse every day. ’lower_case_table_names=2′ configuration setting in Windows is the only reasonable way with MySQL to ensure table naming integrity across platforms (you will not use the name ‘thisissometable’ but ‘ThisIsSomeTable – because the latter is much more readable. Also in any programming language you would use letter-case conventions to make code readable).

Examples on this:
lower_case_table_names is non-functional with VIEWS: http://bugs.mysql.com/bug.php?id=20356
lower_case_table_names is non-functional with BACKUP/RESTORE: http://bugs.mysql.com/bug.php?id=46266

Also here reports have existed for years with no activity.

4)
Existing GUI tools are effectively discontinued before something else is properly in place. Most MySQL Administrator bug reports get classified as “won’t fix” because every feature in MA is planned to be migrated into Workbench. Same applies to Query Browser, but that worries me less as I have never really been able to use that program a meaningful way. The MA Tray Applet in particular is almost indispensable for Windows users developing and testing an application that is supposed to work with more MySQL versions and different configurations. Now Windows users are left in a vacuum. Existing software deteriorate, new one not functional (frankly in my opinion it will take very long time before it will, if it ever will. But Workbench developers of course have more positive expectations). This actually applies to users on all platforms, but it is my clear impression that  the GUI-tools – and MA in particular – are used most extensively by Windows users.

5)
Specifically there seems to be quite a lot of issues that are related to 64 bit Windows. Some of the issues in category 2) may be and with issues in category 4) it is particularly annoying that the MA tray applet start/stop/configure service does not work on 64 bit systems. It simply fails to connect to Windows Service Manager on such systems.

http://bugs.mysql.com/bug.php?id=44993
http://bugs.mysql.com/bug.php?id=45050

Also in http://bugs.mysql.com/bug.php?id=44993 I had the reply “Windows 7 is an unsupported platform”. True that Win7 is not listed in the manual as a supported Operating System, but is OpenSuSE 11.2? Could you imagine a report with OpenSuSE 11.2 (that is not even an alpha – just a ‘developer milestone’) being rejected because of being “an unsupported platform”? Never!

Fact is that Win7 is now shipping and before the end of this year it will probably be the dominating Windows Operating System. And in relation to the matter that was discussed, there is no difference between Vista and 7. I refuse to believe that this is neither ignorance or just stupid formalism. It is a bad excuse only for doing nothing (and I do not blame the supporter, because I do not believe she took the decision herself).

Wake up to reality please! Win7 64-bit is the state-of -art Windows OS. MySQL missed the Vista train (it took almost one year before MySQL took Vista seriously) – don’t miss the 7-train because of sleeping too long!

6)
Miscellaneous issues affecting Windows but not most Unix:
http://bugs.mysql.com/bug.php?id=31109
http://bugs.mysql.com/bug.php?id=43184
http://bugs.mysql.com/bug.php?id=43273
http://bugs.mysql.com/bug.php?id=45996

activity? nonono …

It is disputable of course how serious those are. Anyway one of those was a complete ‘showstopper’ for me some months ago. But that is not the point here.  The point is that every time (some) MySQL people observe that Windows cannot be fully controlled from a ‘cmd.exe’ prompt or a  Cygwin console they are overwhelmed by surprise!

So those were the symptoms, but what are the reasons?

A MySQL developer wrote to me lately “Mostly I develop on Linux. Sometimes I have to test on Windows. I never cared about [some specific configuration option] and never had problems. The main problem may be that most MySQL developers share this experience. Recently I was asked to develop cross-platform tests [..]. Someone mentioned that [the above mentioned configuration option] should be tested too. This was the moment when all the problems started. I found a bunch of problems. Partly ridiculous simple oversights. I found most were reported as bugs already …” and he advised that I should for instance blog about this to get attention because the bug reports only or mainly affecting Windows users are effectively ‘on ice’.

These are the culprits of the problem:
1) Unit-testing on Windows is not done during development. Development happens on Linux (and lately Solaris) and then ‘we’ can just afford a few hours to check on Windows. It fails of course because it is too late.
2) Also there seems to be no system that overall management guidelines have effect in the organisation of MySQL as regards to considering Windows important. People are allowed to go on with the attitude ‘should I really care about windows’. I do not claim that everybody has this attitude – not even the majority.  But if only a few core people have it is enough to be destructive.
3) Finally I have also noticed that supporters replying to bug reports will often not install MySQL as a service but start it as a user program instead. If the issue is that “SELECT 7″ returns ‘8′ instead of ‘7′ that of course does not matter. But for other reports it may do. For instance this one:
.. what is only reproducible when MySQL runs as a service with system privileges (and no doubt .. I myself was just as stupid as the MySQL people involved!) . That would have taken 2 minutes to sort out only if MySQL people use Windows (and even MySQL software for Windows) as Windows users do. But they treat Windows as an OS that is managable from ‘cmd.exe’ just like Unix is managable from a console or shell.  It is not the case.

Sometimes complete ignorance about Windows is exposed. One such example (no link and no name mentioned): A user reported some issue that MySQL would not start as a service on Windows. A MySQL person replied “On FreeBSD the script that starts the server does like this …”.

Sigh!!

19 thoughts on “Does MySQL care about Windows users?

  1. Peter,
    thanks for bringing attention.
    Imn my opinion, the core problem with the MSI installer is that most developers even if they are Windows-savvy do not have a clue about how installer is created. The process of creating the installer is a magic, the sources for installer are not checked into the server repository, and it is not built when mysql is built. Hence, it is also not tested until a release is done and if some bugs creep in it is normally too late to fix them.

    I would not comment on installer bugs further, not my area of expertise or responsibility, sorry. Yet, I’m completely aware that it has to be nearly perfect and problems here make users/customers shy away. I also think that we cannot afford to tell people – use the binary (zip) distribution, as this thing is completely foreign in Windows world.

    Having said this, I believe it is relatively simple to write some scripts what would do everything installation does, but for zip distribution. It is not much, e.g create a service with “sc.exe”, run it under reasonable user account (*not* SYSTEM) for security reasons, put data and config file into reasonable place where you can edit it . Edit config manually, without the need to start an *elevated* ConfigWizard. Would this improve your own experience?

    >But they treat Windows as an OS that is managable from ‘cmd.exe’ just like Unix is managable from a console or shell.

    I’m afraid it is worse:) Windows is only manageable with bash via ssh from Linux desktop(one needs to install cygwin sshd service on the windows box of course ;)

    >“On FreeBSD the script that starts the server does like this …”.
    can’ stop laughing here:)

  2. As you allude to, the problem is between management and developers. Where is the middle management, whose job it is to translate the directives from above into directives below?

    Do the developers have complete freedom to work on the bugs they choose? Are bug fixes completely at the mercy of the whims of the developers? I doubt it.

    Many of the issues you describe above are similar to issues on Linux too — there are serious bugs all over the place, but they need “enough” people to complain (I’ve found that once three people complain, they take another look at it). So post your “me too” posts to bugs, because they really do help.

    The fact that most developers don’t use Windows is a *symptom* of the problem, not the cause. Someone needs to say “these 20 bugs need to be worked on this month” and assign the bugs to developers, if they are not chosen by anyone. The MySQL development team needs more structure, which I’d hoped Sun (and now I hope Oracle) would provide.

  3. I was one of the few server developers who regularly did development work on Windows. I even had a dedicated Windows development machine for the purpose.

    One of the frustrating things of my daily work was fixing the Windows build because foo-developer was too cool to fix the breakage they had caused because it worked fine on Linux. Another problem was the lack of due care and attention that other platforms, other than Linux, may exist; to the degree that even platforms such as FreeBSD were at a disadvantage.

    One of the things I have been contemplating is releasing binaries/patches for having storage engine plug-ins on Windows. This was first demonstrated within MySQL as far back as 2007 – my proof-of-concept binaries are attached to the relevant worklog. What I need to do is learn the mysteries of the MSI installer stuff, or simply find someone willing to do the packaging.

    Alas, many developers simply don’t care.

  4. > 1) Unit-testing on Windows is not done during development. Development happens on Linux (and lately Solaris)
    > and then ‘we’ can just afford a few hours to check on Windows. It fails of course because it is too late.

    This is untrue. Every push to development trees gets tested on quite a few Windows hosts. Whatever the developer uses as his development machine is not very important, once the code gets pushed to a development tree it gets tested on a myriad of platforms.

  5. My prediction is that after the merger with Oracle MySQL will care much more about running on Windows. Hopefully they will have a few developers who don’t mind concentrating on that platform.

    I have no idea how performance changes for InnoDB on Linux versus Windows. I expect that there are a few significant differences but I have yet to find benchmark results for it.

  6. Having worked with a couple of installers in my days, MSI always seemed like black magic to me.

    I’d rather make my own assumptions, rather than having the MSI tell me I can’t have 2 versions of the same product installed.

    Been digging NSIS for a couple of years now though.

  7. I’d like to clarify Davi’s point:
    yes, every push gets tested on myriads of platforms (miryads being actually 5 : Windows, Linux, Solaris, Mac OSX and FreeBSD, whereby those OSes run on at least 2 different hardware architectures). The environment in which server is tested does not resemble customer environment: it is just testing of just compiled stuff with mysql-test-run.pl.

    MSI installation is not unit tested (I *believe* neither Linux RPMs nor OSX DMGs are tested). Installer is smoke-tested by volunteers, when very-soon-to-be-released binaries are available.

  8. “every push gets tested ..” Exactly, and that is the problem because it is too late when it is already pushed! No automated test suite can replace an attentive developer who knows what he is doing with the platform he is working on. Test suites are designed to find *known issues* and *issues that are possible to foresee with our current level of imagination* only!

    New code should be tested on most important platforms during development and before being “pushed”.

    A very good example where this has failed completely is the missing support of Windows file system Unicode support in both server and client (see two of the links I posted). Nobody *foresaw* this (and every MySQL person invoved in that discussion had a complete surprise because they normally use a command-line when using Windows and never realized before that Windows Unicode functionalities are not available from command line – only from Windows GUI and the Windows API) and as a consequence no test suite ever caught it either. Nobody *foresaw* this and nobody ever created a test for such issues for the same reason.

    “Unaccountability” I think would be the English term when a developer lets a test suite do the thinking for him. Test suites do not think!

  9. I’m not sure how you manage to have any traffic at all to a Windows server. There’s a bug which pretty much limits you to a few hundred connections:

    2048 file descriptor limit on windows needs increasing
    http://bugs.mysql.com/bug.php?id=24509 (Fixed in 6.0+)

    I disagree with you on pushing the testing requirements onto the developer though. It depends on the definition of ‘pushing changes’. If it pushes into a team tree before trunk, then that is probably a better place to apply QA for new features. The server doesn’t need more beaurocracy in its development process, and I don’t think you can expect all developers to perfectly understand all platforms.

    - Morgan

  10. re. Windows unicode file system it was not a complete surprise for everyone involved:) Supporting Unicode filesystem on Windows and encoding-less file names everywhere else is not as simple as one might think. I do not think it is absolutely necessary for a DBMS to handle Unicode in filenames.
    It is nice, but one can live without. does one need unicode table names? I doubt. Unicode data in tables- no question, yes. Correct sorting, of course.

    But filesystem? use case of being able to LOAD DATA INFILE ‘ИНФАЙЛ’ on danish windows is not strong enough to do major rework, IMO.

  11. A few posts require a short reply here.

    1)
    I did not claim that unicode support with file system is priority-1. But it is very obvious that developers/team/supporters were surprised. Various platforms have different unicode implementations and different file systems have different case-sensitivity and everybody should know of course. Personally I do not care much about this issue server-side but client-side it is a problem.

    And btw: the letter-case issues affect OS-X (on default file system) as well.

    Actually I forgot one thing in my post post, and that is that command-line client on Windows is less functional on Windows that on *nix. Since MySQL 4.1 was released with support for unicode data, ‘cmd.exe’ never was an appropriate “framework” for the client. I think there is no other way than building a special client for windows with the Windows API – and that applies to both a command-line client as well as a libmysql-type client that can be compiled or included into applications.

    2)
    Of course I do not expect everything fixed overnight. But I would prefer clear and realistic plans communicated. What is considered important, what is not and what can we expect and when? We all need that info – also to decide if MySQL is the right choice for a particular project or not. If there is some consensus about this among MySQL developers and their management it is not clear what it is. I think this is also what Sheeri means with ‘more structure’.

    3)
    Whatever problems there are with a large number of connections to a MySQL server on Windows is not very relevant for the discussion in my opinion. My scenario is production server on Linux, application development on Windows (and that definitely makes sense because of the ease of managing multiple server instances on Windows). But that is also now partly broken. It is not as easy now as it used to be.

    4)
    That installers is not under version control at all and not considered a fully integral part of the product/program is horrible in my opinion. I did not realize it was so bad. But I never had issues with RPM’s on SUSE (what I also use from time to time) – installers for Windows seem to be in a very much worse state than RPM’s and DMG’s. My complaint was that obivously nobody cares. I saw this reply frequently “we cannot reproduce at our end”. However ability to reproduce depends on what effort you put in it!

  12. “My prediction is that after the merger with Oracle MySQL will care much more about running on Windows. Hopefully they will have a few developers who don’t mind concentrating on that platform.”

    This would be the same Oracle that runs all of its internal infrastructure on Linux, would it? The same Oracle that already has a rather well regarded Database offering?

    I wouldn’t hold my breath if I were you.

  13. The open source development model has traditionally been about developers providing source code, and others (typically OS distributors) providing binary distributions.

    Tray applets and GUI tools are typically far, far removed from the thoughts of most system admin and server software developers. If you’re so displeased with the current state of the Windows installer, by all means pitch in and help the project along. If you insist on good Windows support, I found PostgreSQL/pgadmin to work extremely well when I installed them on a windows box at my work.

    The project has clearly grown and done rather well on its own, despite weak Windows support, so I don’t think they’re hurting because of all the Windows users who find installing MySQL more trouble than its worth.

    Also, I would be careful about placing too much weight on the number of downloads based on various platforms. I’ve never downloaded MySQL sources or binaries directly from MySQL.com, and I don’t think that’s how most people get the software. Back when I considered myself a MySQL user I used either the binary that came with my linux distribution or downloaded a srpm and compiled it from source.

  14. Pingback: Mark Clarke (mxc) 's status on Saturday, 25-Jul-09 10:22:59 UTC - Identi.ca

  15. Pingback: Log Buffer #156: a Carnival of the Vanities for DBAs | Pythian Group Blog

Leave a Reply

Your email address will not be published. Required fields are marked *

You may use these HTML tags and attributes: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <strike> <strong>