Last week I completed the refactoring of function files. I moved each function into a class as a static method. This allows us to use the autoloader to load up the class instead of using a file include.
Last week I worked on converting function files into class files and I changed the function calls to static method calls. This allowed me to remove the include calls in favor of the autoloader, that loads the classes automatically.
Last week was pretty productive on bug fixing side. These were quite varied from server monitor breakage on non Linux and Windows platforms to minor rendering issues of date selector. Quite interesting new feature is that our SQL parser now better understands to some MariaDB extensions.
Last week was again mostly spent on fixing bugs and going through reports on our error reporting server. This is really valuable tool for finding problems which hit our users, on the other side there is simply too many things there and sometimes it's hard to pinpoint something really important.
Besides that I've also ensured that all our libraries work well with upcoming PHP 7.2 and all are now tested on Travis CI against this version as well.
It will add the feature to have an easy view of the JSON data of a column of a table. Right now it shows in only one line, without any kind of format, so we can have an option to show when you are viewing a table with JSON data or when editing a nice JSON view.
There are several places which uses inline javascript (like onclick, onsubmit and onchange). These should be removed and placed into javascript files. After removal we can get rid of ‘unsafe-inline’ for scripting in CSP.
If the import file is not in supported formats and it’s not ‘utf-8’ then it runs the SET NAMES for the selected charset during the import option. So I will disable the second time encoding if the selected charset is same as that of the charset mentioned in the SET NAMES.
It would be really nice to have a progress bar showing import and export progress in real time in terms of percentage and having some more detailed information like which step/table it is processing.
For Export (export.php will be used)
Export Status in Percentage (including how many tables and rows have been already processed and which is in progress right now)
For Import (import.php will be used)
Upload a file and Unzip it. Import into a database (including how many tables and rows have been already processed and which is in progress right now)
When a user has access to a db (or table), but is not in the user table he’s shown as not having a password. Even after successfully running the query delete from mysql.user where user = ‘test’ I was still able to login with the test account.
I will show the message “NO” without red mark and with showing an information icon with a tooltip saying that the user exists in some tables but not the user table.
The PMA should fallback to default table names if controluser + pmadb options are set while other like ‘relation’, ‘column_info’ are NOT in config.inc.php.
So if the above conditions are matched I will make all features that are listed as Enabled instead of Disabled.
I have learned a lot of new things. This organisation have the best community I have seen it till now. I will keep on contributing to this organisation to make phpmyadmin better.
Thanks phpyadmin for the awesome summer and special thanks to my project mentor Isaac Bennetch :)
This post summarizes the work done and tasks accomplished during the last twelve weeks. It was carried out as a part of Google Summer of Code 2017 with phpMyAdmin.
The project involved two separate sub-lists of tasks, one related to phpMyAdmin’s Error Reporting Server and the other related to phpMyAdmin’s selenium test-suite.
The project led to a few good improvements to phpMyAdmin’s error reporting server, which I hope would make it more developer-friendly and the public interface would help more non-team contributors in fixing the errors reported on it.
phpMyAdmin’s selenium test-suite has finally stabilized in green (i.e. passing), albeit some errors popping up once in about ten runs. I do hope to fix these as they are encountered. Improvement in selenium test-suite and moving it out of allowed failures in Travis has already started to show its impact as we could track a few errors introduced while refactoring. These would otherwise be missed as the refactored code was not well-covered by the unit tests.
Finally, it has been another wonderful summer working with phpMyAdmin and its wonderful community. I would like to thank my mentor, Michal Čihař and the entire phpMyAdmin team, for their continued support throughout this project.
Hosted Weblate provides also free hosting for free software projects. The hosting requests queue has grown too long, so it's time to process it and include new project.
This time, the newly hosted projects include:
Gadgetbridge - cloudless replacement for your gadget vendors' closed source Android applications.
SUSI AI Chat - artificial intelligence for personal assistants, robots, help desks and chatbots
If you want to support this effort, please donate to Weblate, especially recurring donations are welcome to make this service alive. You can do them on Liberapay or Bountysource.
Add selenium tests to cover Query-by-example UI on phpMyAdmin
Made selenium test-suite more robust
Key tasks stalled:
NA
Tasks in the upcoming week:
Details:
I completed one of the tasks left in the proposal that was about adding selenium tests for covering phpMyAdmin’s Query by example UI. Additionally, I continued to try and make phpMyAdmin’s selenium test-suite more robust.
Another GSoC student Raghuram has been working on developing the new UI for Query by example as a part of his project. His PR with those changes just got merged yesterday, but the selenium tests might have to be adjusted to suit this new UI. Apart from this, the selenium test-suite is on the verge of running successfully over a set of continuous commits.
With this week coming to an end, the offical coding period for GSoC ’17 has also ended. I would be discussing with my mentor Michal to decide on what to submit as the final submission. It would mostly be a summary post with relevant links and description on this blog.
Since some time we're using siphash algorithm to speed up looking up strings in Weblate. Even though it is used by Python internally, it's not exposed in the standard library so several third party modules appeared in the PyPI. Out of all these siphashc or rather it's Python 3 fork siphashc3 seemed to perform best, so I've started to use that.
After trying to get fix into siphashc3 without much of success, I've spoken to original author of siphashc and he has agreed to hand over maintainership to me. So it's new home is at https://github.com/WeblateOrg/siphashc and new release is already available on PyPI.
Note: Originally we were using MD5 in Weblate, but siphash has shown to be faster and fits into 64-bits, what makes it easier to store and index in SQL databases as LONGINT.
So, finally I made it to the end of my project. It was a great experience working with phpMyAdmin through the summer. Since final evaluations are drawing close, Isaac suggested me to make a blog post of links to all the pull requests I submitted. That makes this post an official artifact of my GSoC project. I am glad that I was able to stick to the proposed plan for the most part. Towards the end, we had to re-plan slightly. I am thankful to my mentor for being very supportive during this time.
The following is the list of Pull Requests I submitted in chronological order:
Last week was a bit less intensive on bug fixes, but I've spent quite a lot of time on improving our localized documentation and fixing various errors there. This is really something what should be automatically checked in Weblate in the long term, but right now it results in build failures in Travis.
Previous week the tasks mentioned in my proposal was completed so this week was mostly spent in debugging, checking and fixing other issues. I have fixed one more issue this week.
Add selenium tests to type SQL query and execute it on phpMyAdmin
Key tasks stalled:
NA
Tasks in the upcoming week:
Details:
I spent some time last week to make the test-suite more robust. Last I had written it was throwing a lot of unrelated errors, but that has been fixed now. It felt really good to see the ‘Daily update’ mails from Browerstack for last two days which said ‘No builds had issues’, which seems to be pointing to suite been stabilized for a few consecutive builds.
Last week was spent at DebConf’17 in Montreal, Canada. Madhura, Michal and I were attending the conference and we managed to meet up with Marc for lunch on one of the days. I did manage to find some time and added selenium tests for typing and executing SQL queries. This covers the Server, Database and Table SQL pages and in-line editing of SQL queries on results page.
For this current week, I plan to continue working to making the selenium test-suite consistent to pass on every commit and prevent any non-deterministic errors. This coming week would be the last week working before pencils-down date. I do expect to continuing some work towards the test-suite to surely go on.
I've spent last week on DebConf, where main motivation to come was to meet with other phpMyAdmin team members. Besides that we've met with Marc, former phpMyAdmin lead.
On the coding side, it was pretty much just reviewing pull requests and some minor fixes.
Hosted Weblate provides also free hosting for free software projects. The hosting requests queue was over one month long, so it's time to process it and include new project.
If you want to support this effort, please donate to Weblate, especially recurring donations are welcome to make this service alive. You can do them on Liberapay or Bountysource.
First of all, I successfully passed my second evaluations. Thanks to my mentor.
The reason I didn’t blog since then is that there was not much progress. Reopening of my college and registration took up more time than expected. Adding to the trouble, my next task Setup improvements turned out be more challenging than expected. It was not suggested in the initial project enhancements collection. But I added it as I felt it is important feature and a good replacement for the other task – Filling random test data in database. Why I had to replace? That’s another story.
Coming to the point, allowing access to file-system is a sensitive issue. After a lot of discussion with Michal, we finally agreed on following scheme for allowing access to setup:
If there is no config.inc.php and it can be written to, redirect user to setup and force him creating config at least with setup password. This will make the attack window minimal as most people will try to access the tool just after installing.
For existing installs user has to manually add the password to the configuration. This is needed to avoid somebody remotely creating that.
Access to setup:
If there is no config file, access to setup is allowed for initial setup.
If there is config without password, access is rejected with link to documentation how to enable it.
If there is config with password, user has to enter the password prior entering setup.
And this was finalized just last week. Also, I came across a few existing bugs after starting to work on setting up the credential-setup. These bugs further added to delay. I had to fix them and proceed. Current standpoint is that credential-setup is still not completely done. There is one more issue I ran into. The config options which I applied are not reflecting in the generated config file. I and Isaac are working to resolve this. You can check out work in progress by checking out my branch. Now it is clear that my idea for Setup Improvements was ambitious. I talked with Isaac about this. He understood the situation as the pencils down time is approaching. Now I plan to just implement editable configuration completely for this task.
Weblate 2.16 has been released today while I'm at DebConf17. There are quite some performance improvements (and more of that is scheduled for 2.17), new file formats support and various other improvements.
Full list of changes:
Various performance improvements.
Added support for nested JSON format.
Added support for WebExtension JSON format.
Fixed git exporter authentication.
Improved CSV import in certain situations.
Improved look of Other translations widget.
The max-length checks is now enforcing length of text in form.
Make the commit_pending age configurable per component.
Various user interface cleanups.
Fixed component/project/sitewide search for translations.
You can find more information about Weblate on https://weblate.org, the code is hosted on Github. If you are curious how it looks, you can try it out on demo server. You can login there with demo account using demo password or register your own user. Weblate is also being used on https://hosted.weblate.org/ as official translating service for phpMyAdmin, OsmAnd, Turris, FreedomBox, Weblate itself and many other projects.
Should you be looking for hosting of translations for your project, I'm happy to host them for you or help with setting it up on your infrastructure.
Further development of Weblate would not be possible without people providing donations, thanks to everybody who have helped so far! The roadmap for next release is just being prepared, you can influence this by expressing support for individual issues either by comments or by providing bounty for them.
This week was quite productive in terms of bug fixes and various code improvements. One of most visible things is probably PHP 7.2 compatibility, but there were other fixes as well.
I've somehow failed to post this report on time, so sorry for late publishing.
First of all if you are still using nijel/weblate, you should switch to weblate/weblate. They both currently share same configuration, but it might happen that some future updates will go to the weblate owned container only.
Now back to the container changes. Since beginning we were using Django built in server. That's fine for development purposes, but it really doesn't work that well in production as it can handle only one request at time. Therefore we've switched to more robust approach using nginx + uwsgi + supervisor.
Thanks to this, the docker-compose no longer needs separate nginx server as everything is now sanely handled within the weblate container itself.
After fours years, I will again make it to DebConf, I'm looking forward to meet many great people, so if you want to meet and happen to be in Montreal next week come and say hello to me :-).
It seems I've settled down on four year schedule - I've attended DebConf09 and DebConf13 so far. Let's see if next one will come in 2021 or earlier.
Implemented marking of test failures on Browerstack
Researched over how to use php-webdriver
Key tasks stalled:
NA
Tasks in the upcoming week:
Details:
I spent some time last week to make the test-suite more robust and it actually did stabilize for a few consecutive builds. I also spent some time to add code in our test-suite so that the failed tests on Travis are actually marked with a failed status on Browerstack.
For this current week, I plan to continue working to making the selenium test-suite consistent to pass on every commit and prevent any non-deterministic errors. For the last two days, it has started to throw a lot of errors suddenly (which I am really not sure why it is happening). I might also start to implementing the new selenium tests to be added for typing and executing a sql query on table, database and server SQL pages.
Weblate 2.16 is almost ready (I expect no further code changes), so it's really great time to contribute to it's translations! Weblate 2.16 will be probably released during my presence at DebConf 17.
I'd especially like to see improvements in the Italian translation which was one of the first in Weblate beginnings, but hasn't received much love in past years.
Fixed selenium tests related to Export, Table, Tracking, XSS
Key tasks stalled:
NA
Tasks in the upcoming week:
Make selenium test-suite run consistently (i.e. prevent timing issues non-deterministic errors)
Explore the possibility of migrating to use php-webdriver by Facebook
Details:
Last week was cut short since I was travelling to attend my convocation ceremony at the BITS Pilani, Pilani campus. I graduated in the first division with a dual degree in M. Sc. (Hons.) Economics and B. E. (Hons.) Computer Science.
I spent the last week fixing the selenium tests related to Export, Table, Tracking, XSS. The related PR was made and merged at #13520. The test-suite is able to run successfully and finally I got a few successful selenium travis jobs after a long time, though I am still fixing some or the other error that keeps popping up on some jobs.
For this current week, I plan to continue working to making the selenium test-suite consistent to pass on every commit and prevent any non-deterministic errors. I would also want to explore a possibility to migrate the selenium tests to use Facebook’s php-webdriver.
Last week was really focused on fixing issues on phpMyAdmin itself. Some of them also lead to me to bigger cleanups, for example in theme management code, which is now not relying on session cache.
Another important improvements were focused on improving SSL support in phpMyAdmin. It is now able to automatically detect if server enforces SSL and enable it in such case. There is also improved documentation about configuring SSL.
All are probably familiar with vpn.iiit.ac.in. It clearly details steps to follow for setting up IIIT VPN which includes going through a series of steps every time you want to connect. So, I made this small procedure to automate it.
Follow first two steps on vpn.iiit.ac.in for Linux. Here are the two steps:
Install openvpn package using yum, apt-get, synaptic, yast etc. based on which Linux distribution you are using. All standard Linux distributions support openVPN installation using standard repositories.
After installation, copy following files to /etc/openvpn/ directory. ca.crt, all.iiit.ac.in.crt, all.iiit.ac.in and linux_client.conf. (You need to be root to be able to write in this directory).
cd into etc/openvpn/. Create a file called login.conf and add your IIIT email ID in first line and password in second line. For security, make it accessible only to root:
Open linux_client.conf. Find line which says auth-user-pass(should be around line 126). Replace the line with auth-user-pass login.conf.
Go to home directory and open .bashrc. Copy the following lines at the end of it.
#This function is helpful if you want to run any function as root.
function execsudo ()
{
### ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ ##
#
# LOCAL VARIABLES:
#
### ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ ##
#
# I use underscores to remember it's been passed
local _funcname_="$1"
local params=( "$@" ) ## array containing all params passed here
local tmpfile="/dev/shm/$RANDOM" ## temporary file
local filecontent ## content of the temporary file
local regex ## regular expression
local func ## function source
### ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ ##
#
# MAIN CODE:
#
### ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ ##
#
# WORKING ON PARAMS:
# ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
#
# Shift the first param (which is the name of the function)
unset params[0] ## remove first element
# params=( "${params[@]}" ) ## repack array
#
# WORKING ON THE TEMPORARY FILE:
# ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
content="#!/bin/bash\n\n"
#
# Write the params array
content="${content}params=(\n"
regex="\s+"
for param in "${params[@]}"
do
if [[ "$param" =~ $regex ]]
then
content="${content}\t\"${param}\"\n"
else
content="${content}\t${param}\n"
fi
done
content="$content)\n"
echo -e "$content" > "$tmpfile"
#
# Append the function source
echo "#$( type "$_funcname_" )" >> "$tmpfile"
#
# Append the call to the function
echo -e "\n$_funcname_ \"\${params[@]}\"\n" >> "$tmpfile"
#
# DONE: EXECUTE THE TEMPORARY FILE WITH SUDO
# ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
sudo bash "$tmpfile"
disown 2> /dev/null
rm "$tmpfile"
}
function vpnfunc {
echo "Starting VPN!!"
cd /etc/openvpn
openvpn --config linux_client.conf | while read line
do
[[ "${line}" == *"Initialization Sequence Completed"* ]] && echo 'nameserver 10.4.20.204' | cat - /etc/resolv.conf > temp && mv temp /etc/resolv.conf && echo 'Started VPN successfully!!' && kill -STOP $$
done
}
function startvpn {
exec 3>&2 # 3 is now a copy of 2
exec 2> /dev/null # 2 now points to /dev/null
execsudo vpnfunc # run script with redirected stderr
exec 2>&3 # restore stderr to saved
exec 3>&- # close saved version
}
function stopvpn {
sudo killall openvpn
sudo sed -i '/nameserver 10.4.20.204$/d' /etc/resolv.conf
}
Open a new terminal or execute source .bashrc.
From now, startvpn command should start your VPN. (It may ask for sudo password).
stopvpn will terminate VPN.
I am not a bash God, so please spare me if I made any blunders. But, do leave a comment if it can be done better.
First I made changes in the phpmyadmin configuration sql table and added one extra table to track the progress status.
CREATE TABLE IF NOT EXISTS `pma__progress` ( `uuid` varchar(36) NOT NULL, `type` varchar(64) NOT NULL, `data` text NOT NULL, `value` int(50) unsigned NOT NULL, `total` int(50) unsigned NOT NULL, UNIQUE KEY `uuid` (`uuid`) ) COMMENT='Saved import/export progress data' DEFAULT CHARACTER SET utf8 COLLATE utf8_bin;
This week, there is nothing much to update. It turned out that my previous task i.e., to consolidate table sorter libraries used by phpMyAdmin is not needed. The reason being the two libraries mentioned in the task are independent of each other and do not do the same task. I am getting code reviews on my previously submitted patches. I will be working on them those patches this week and hopefully get some of them merged.
Having publicly running web application always brings challenges in terms of security and in generally in handling untrusted data. Security wise Weblate has been always quite good (mostly thanks to using Django which comes with built in protection against many vulnerabilities), but there were always things to improve in input validation or possible information leaks.
When Weblate has joined HackerOne (see our first month experience with it), I was hoping to get some security driven core review, but apparently most people there are focused on black box testing. I can certainly understand that - it's easier to conduct and you need much less knowledge of the tested website to perform this.
One big area where reports against Weblate came in was authentication. Originally we were mostly fully relying on default authentication pipeline coming with Python Social Auth, but that showed some possible security implications and we ended up with having heavily customized authentication pipeline to avoid several risks. Some patches were submitted back, some issues reported, but still we've diverged quite a lot in this area.
Second area where scanning was apparently performed, but almost none reports came, was input validation. Thanks to excellent XSS protection in Django nothing was really found. On the other side this has triggered several internal server errors on our side. At this point I was really happy to have Rollbar configured to track all errors happening in the production. Thanks to having all such errors properly recorded and grouped it was really easy to go through them and fix them in our codebase.
Most of the related fixes have landed in Weblate2.14 and 2.15, but obviously this is ongoing effort to make Weblate better with every release.
Last week was quite busy and that can be seen from number of issues. Some of them are coming from our error reporting server, where I've focused on the most frequently happening ones for last releases. Still there is about 30000 reports to handle there.
There were several fixes to our SQL parser as well, apparently it's already being used by some other tools, for example by php-sqllint, so we're getting more bug reports :-).
Fixed selenium tests related to Login, Normalization, ChangePassword, ServerSettings
Key tasks worked on:
Fixed table-related, tracking-related and export-related tests (partially)
Key tasks stalled:
NA
Tasks in the upcoming week:
Work on and fix remaining set of broken selenium tests
Details:
I spent the last week fixing the selenium tests related to Login, Normalization, ChangePassword, ServerSettings. The related PR was made and merged at #13476.
In the later part of the week, I spent some time fixing the table-related tests, tracking-related tests and export-related tests. Though, they are not in a consistently fixed state (i.e. they sometimes do misbehave, leading to a failure), I expect them to be ready for a PR soon.
For the next week, I plan to continue working to fixing the next set of broken selenium tests.
I tried solving this issue by setting the cookie and the retriving the cookie to show the progress but the setcookie() in php failed when used after setting the header.
So as this doesn’t worked I have now created a new table in phpMyAdmin Configuration Storage database. I am running behind my timeline as this seems to be difficult compared to my expectations.
This week, I worked on adding two-factor authentication support for phpMyAdmin. This is by far the most interesting feature I worked on. I never really worked keenly on the security aspect of any software. This indeed has been a very good learning experience.
Here is the process:
Log in to your account normally.
Access setup2FA.php. I have not figured out where to place the link which directly takes you to the page. That is more of a user-experience issue and I leave it for the team to advise me on that.
Open Google Authenticator (or Authy or whatever you prefer). Scan the barcode with your app. It now starts generating TOTP on your app.
Enter the TOTP in the text field and click submit. Done!!. You now have successfully added 2-factor authentication to you PMA account.
When you log in next time, after you enter your credentials, you will be asked for TOTP.
Enter the TOTP generated. You will not be logged in unless you clear this step. That’s it!! :D.
I am yet to add a way to delete 2-factor authentication. You can check-out the code from my branch. I will make a pull request after I add the deletion part also.
Since this is a security feature, I feel this requires thorough testing.
Improved the test coverage for the Error Reporting server from 47% to 83%
Fixed selenium tests related to CreateDropDatabase, CreateRemoveUser, Database-related operations
Key tasks stalled:
NA
Tasks in the upcoming week:
Work on and fix next set of broken selenium tests
Details:
Based on Michal’s review in the first GSoC evaluations, I worked on improving the test coverage of the Error Reporting Server codebase. The code coverage has improved significantly from 47% to 83% during this week. It now covers most of the important functionalities and we should be able to catch any breakages due to code changes in future.
Apart from that, I started on the second part of my proposal which deals with fixing the broken selenium testing suite in phpMyAdmin. I have fixed around fifteen tests of around fifty five tests which we have. I have made a PR at #13462. Once all the selenium tests are fixed, I plan on adding a few more to cover some other important actions that have not been covered till now.
For the next week, I plan to continue working to fixing the next set of broken selenium tests.
I am late to update the blog this week. Sorry for that.
First, I want to talk about first evaluations whose results came out on 30th. I have successfully passed the evaluations. My mentor gave me a very postive feedback which is really motivating.
Few updates about past work:
Multi-table query UI: When I submitted this patch, it had a few problems about which I notified my mentor. He helped me fix them and I believe the patch is almost ready for merge. We also decided to remove old code in a different pull request.
Allow designer to show other Databases: This is the issue I am working on currently. I must say this is what also caused my delayed posting. Earlier this week, while meddling with some server-related stuff, I accidentally lost my repository. I was able to retain all the branches I pushed online. Since I didn’t push this one, I was unable to recover my work. Anyway, I am able to catch up after that. Learnt a lesson to push the code as soon as I am done for the day.
Cutting to the chase, here is the screenshot:
So, as discussed, I added an option in designer side menu (I copied some other icon since we didn’t decide on one) which on clicking opens a dialog. You can select the db and the table you want to add. When you submit it, your new table is added. You can create relationships like any other table and the created relationships get stored. This is the basic functionality and it is done. The problem is that once the page is saved or reloaded, only the tables in current db appear back. According to the solution we decided, we need to show the tables in other db to which there are relations by default from the beginning. That should solve everything. This is not a big problem and I will fix it by the weekend.
Last week was really about solving bug and pull request. I've managed to go through many of long pending pull requests and most of them were merged either directly or with additional fixes.
I always feel bad when it takes too long to merge pull request, but most of them were actually waiting for some fixes which didn't arrive and I had to fix them on my own. This is often what happens to GSoC students pull requests once they realize they were not accepted in the end...
While working with this page I found one issue that if there are multiple div which I have to make responsive on the same page then working with div id will failed so I replaced all the div with id=”responsivetable” with class=”jsresponsive”
#159 brings the functionality of updating report status according to the Github issue’s state to which it is being linked to. Thus, if we link the report to a Github issue which is closed, we should automatically update the Report status to ‘Resolved’.
#160 implements a controller action (which should be run as cron job from command line) to synchronise the statues of existing linked reports to statues of the linked Github issues. This action can be (and should be) only run from shell.
#74 implements a read-only public interface which allows non-team developers and contributors to look at the submitted error incidents and reports, though they would not be able to make any changes to the reports such as changing its status, linking/unlinking from a Github issue etc.
For the next week, based on Michal’s review, I would be adding some tests to the Error reporting server so that we add a few more controller/models/shells to the test coverage. I have already made two PRs at #168 and #170 in those directions. Once these (and a few more to come) are merged, I would be moving to starting to fix broken selenium testing in phpMyAdmin.
Weblate 2.15 has been released today. It is slightly behind schedule what was mostly caused by my vacation. As with 2.14, there are quite a lot of security improvements based on reports we got from HackerOne program and various new features.
Full list of changes:
Show more related translations in other translations.
Add option to see translations of current unit to other languages.
Use 4 plural forms for Lithuanian by default.
Fixed upload for monolingual files of different format.
Improved error messages on failed authentication.
Keep page state when removing word from glossary.
Added direct link to edit secondary language translation.
You can find more information about Weblate on https://weblate.org, the code is hosted on Github. If you are curious how it looks, you can try it out on demo server. You can login there with demo account using demo password or register your own user. Weblate is also being used on https://hosted.weblate.org/ as official translating service for phpMyAdmin, OsmAnd, Turris, FreedomBox, Weblate itself and many other projects.
Should you be looking for hosting of translations for your project, I'm happy to host them for you or help with setting it up on your infrastructure.
Further development of Weblate would not be possible without people providing donations, thanks to everybody who have helped so far! The roadmap for next release is just being prepared, you can influence this by expressing support for individual issues either by comments or by providing bounty for them.
Last week was a bit less intensive for me, still there were some bugs fixed.
Most of the time was spent on investigating one report from error reporting server. These errors do not come from our code, but we see them in thousands, so apparently something widely spread. The feature seems to come from DirectAdmin, but I've found occurrences elsewhere as well.
I think this exactly shows why you should upstream your patches - it would get proper upstream review and you would not have to maintain it over years (apparently they have five versions of the patch in their patches directory).
#159 aims to bring the functionality of updating report status according to the Github issue’s state to which it is being linked to. Thus, if we link the report to a Github issue which is closed, we should automatically update the Report status to ‘Resolved’. I have made a PR at #163 which is open for review.
#160 aims to implement a controller action (which should be run as cron job from command line) to synchronise the statues of existing linked reports to statues of the linked Github issues. We could have also implemented a shell to do this task but I could not easily figure out how to use GithubApiComponent (without requiring to do the additional refactoring of moving it to a Utility class). So I added a cron-dispatcher which can be run from command line and takes command line argument to run the specified controller action.
For the next week, I would be making changes based on Michal’s review for on my PRs for the above mentioned tasks. I have already started initial discussion and ground work towards the last task on Error Reporting Server (after which I would be moving to fixing selenium testing in phpMyAdmin) of #74.
I have started working on the phpmyadmin codebase from 22 May, 2017. I have spend 5 days in familiarize myself with the phpmyadmin core functionality and architecture. Below are the details of the issues that I have fixed in first week or in community bonding period.
This summer, I was selected for the prestigious Google Summer of Code ’17 program with the phpmyadmin organization. phpMyAdmin is a free software tool written in PHP, intended to handle the administration of MySQL over the Web. phpMyAdmin supports a wide range of operations on MySQL and MariaDB. Frequently used operations (managing databases, tables, columns, relations, indexes, users, permissions, etc) can be performed via the user interface, while you still have the ability to directly execute any SQL statement.
Introduction
phpmyadmin currently has a large number of open issues (~210 at time of writing this). To maintain and improve phpmyadmin core project the numbers of issues should be lower down. This project aims to resolving the major issues and improvements to be done in phpmyadmin. I have selected a list of issues on which I will be working this summer.
Community Bonding
Before the beginning of the official program period, Google allows students one month to get familiar with the organization they will contribute to, to get familiar with the programming practices, source code, get doubts cleared etc. phpMyAdmin is a PHP project that provides wide range of operations that can be performed via the user interface. The project code is available on GitHub at this link https://github.com/phpmyadmin/phpmyadmin
As the world is going more mobile everyday, so It would be a good idea to make phpMyAdmin responsive, so it works on smartphones and tablets too instead of desktop only.
Currently phpmyadmin have two javascript plugins for table sorting so it’s better to replace it with simply one. I will be replacing jquery.sortableTable.js with jquery.tablesorter.js
It will add the feature to have an easy view of the JSON data of a column of a table. Right now it shows in only one line, without any kind of format, so we can have an option to show when you are viewing a table with JSON data or when editing a nice JSON view.
Also, If the team decides they still don’t want to pursue the responsive interface enhancement, I’m prepared to replace that part of my summer with the solving these issues.
There are several places which uses inline javascript (onclick, onsubmit and onchange). These should be removed and placed into javascript files. After removal we can get rid of ‘unsafe-inline’ for scripting in CSP.
It would be really nice to have a progress bar showing import and export progress in real time in terms of percentage and having some more detailed information like which step/table it is processing.
When a user has access to a db (or table), but is not in the user table he’s shown as not having a password. Even after successfully running the query delete from mysql.user where user = ‘test’ I was still able to login with the test account.
The PMA should fallback to default table names if controluser + pmadb options are set while other like ‘relation’, ‘column_info’ are NOT in config.inc.php.
I will implement this feature by allowing user to drag and drop the existing column from the “structure page” as well as from the navigation tree table column entry, to the “New” column entry in the navigation tree columns section of the target table.
Update 1
I will work on the forked repository and submit the Pull Request as soon it is completed. I will also post the weekly update about the project on medium. This is my GSoC’17 proposal.
Mobile platforms are centered around speed and agility. And yet, building for mobile can sometimes feel clunky and slow. It doesn’t have to be that way.
Google Firebase Team came at New Delhi to show how you to build an app in a day. We can do this by having the application code talk directly to Firebase’s managed back-end services. This means we spend less time on infrastructure and more time on building the features that the users care about. And in the cases where we need server-side logic, use Cloud Functions — Firebase’s scaleable serverless solution.
Cloud storage — To Store and share images, audio, video, or other user-generated content easily with powerful, simple, and cost-effective object storage built for Google scale
Firebase now supports multiple bucket support plus region selection so you deploy your app where your customers are.
Firebase Test Lab — To run automatic and customized tests for your app on virtual and physical devices hosted by Google.
Crash Reporting — To diagnose problems in your mobile app with detailed reports of bugs and crashes.
Performance Monitoring — To diagnose app performance issues occurring on your users’ devices.
Google Analytics — Its free + unlimited and can be used to analyze user attributions and behavior in a single dashboard to make informed decisions on your product roadmap.
Cloud Messaging — To send messages and notifications to users across platforms like Android, iOS, and the web for free.
Remote Config — To customize how your app renders for each user.
Invites — To enable your users to share all aspects of your app, from referral codes to favorite content, via email or SMS.
App Indexing — To re-engage users with their installed apps with this Google Search integration.
Admob — To earn money by displaying engaging ads to a global audience.
This talk was given by Doug Stevenson (Developer Advocate, Google). He gives the information on What we can do with Cloud Functions in firebase.
Notify users when something interesting happens — To use cloud Functions to keep users engaged and up to date with relevant information about an app.
2. Resize image — To take advantage of Cloud Functions to offload to the Google cloud resource-intensive work (heavy CPU or networking) that wouldn’t be practical to run on a user’s device.
3. Perform Realtime Database sanitization and maintenance to block offensive language — With Cloud Functions database event handling, you can modify the Realtime Database in response to user behavior, keeping the system up to date and clean.
He showcased a small demo on how to replace text with emoji using cloud function.
After this talk the Hackathon was open for hacking using firebase.
Test your app using Firebase
This talk was again given by Doug Stevenson (Developer Advocate, Google). In this talk he has given the overview on some more fetaures.
Firebase Test Lab for Android — It provides cloud-based infrastructure for testing Android apps. With one operation, you can initiate testing of your app across a wide variety of devices and device configurations. Test results — including logs, videos, and screenshots — are made available in your project in the Firebase console. Even if you haven’t written any test code for your app, Test Lab can exercise your app automatically, looking for crashes.
Performance Monitoring — To diagnose app performance issues occurring on your users’ devices. Use traces to monitor the performance of specific parts of your app and see a summarized view in the Firebase console. Stay on top of your app’s startup time and monitor HTTP requests without writing any code.
Talk by Kushagra Gour
Kushagra Gour is currently working as a front-end developer at an awesome startup called Wingify building VWO.
Side Projects = More Self Improvement
He has developed a lot of side projects that are available on Github. He has given an overview on some of his side projects.
hint.css — A CSS only tooltip library for your lovely websites.
2. screenlog.js — Bring console.log on the screen.
5. Web Maker — Chrome extension for a fast & offline web playground.
Hackathon
After that 7 teams were shortlisted for presenting their idea on stage for 3 minutes.
After that the judges selected team Decoder as the Hackathon winner. They have made a smart travel application that helps to plan your travel and share it with your friends/family.
What Next
They are coming to Pune (24 June 2017 — Novotel Pune Nagar Road), Bangalore (28 June 2017 — Vivanta by Taj, MG Road) and Hyderabad (1 July 2017 — Hyderabad International Convention Center). So you can register for them here: https://events.withgoogle.com/in-firebase-appfest/
ngDownloader is a web application that gives the download links of the videos from various websites like youtube, facebook, udemy etc with no ads. I have made this application for AngularAttack 2017 Hackathon and selected as winner in Fun/Utility Category.
I have used Amazon AWS Lamda function and API gateway to host and run the backend. Here is the link of the backend code. Also it uses youtube-dl library to fetch download links.
Bugs or feature request ? Drop email at hi@manishbisht.me I would be happy to make it available for you.
I will keep making more improvements and adding features in the future. I am using my Kendo UI Professional license so that I can improve the UI and use more components.
There is the yearly renewal is charge for the domain per year and pay per use charge for Amazon AWS for backend. If you want to contribute a year or pay for hosting, send me an email and I’ll add the years on. Of course I’ll do my best to continue running the domain and hosting, but this is your chance to contribute to the community project.
Talk to me
If anyone would like to get in touch with me I am not too hard to reach. Just drop an email at hi@manishbisht.me I would be happy to have a chat with you.
I submitted pull request for my second task i.e., adding configuration options for default transformations. As I explained in the proposal. I thought I will have to add configuration for every transformation. But I realised not all of the transformations make use of options. So, I added configuration only for those that looked like they make use of paramaeters. I have a feeling that a better option would be to add empty arrays for those transformations that don’t make use of options because in future, the transformations may be modified to make use of parameters and they can modify these empty arrays. Anyway, I raised this question in comments on my pull request. I will act according to response.
I also thought I will have to add configuration options in page-wise settings and setup script separately. But, I realised any change in configuration directives reflects in both setup script and page-wise settings. Awesome!!
Screenshots of new configuration directives:
I will be moving on to my 3rd task this week – Allow designer to show other Databases. According to my plan, I will be submitting code for my 3rd task after first evaluation, but I have to start discussing design with my mentor now. I guess this puts me in a good position for my first evaluation as I submitted all required code. From now on, I will be addressing reviews on the submitted code and plan for my 3rd task.
Besides usual bug screening and pull requests merging, I've spent quite some amount in digging reports in our error reporting server and fixed the most visible ones.
The error reporting server collects errors happening in phpMyAdmin installations worldwide (this is opt-in reporting) and gives us insights where our users suffer most. Some of the errors are really weird and probably indicate PHP bug, but as we don't collect more information than is necessary, we really can not say for sure and we can not find person to reproduce the bug. Anyway if something has happened hundredth times on several installations, it's probably worth fixing in our code base.
#98 aimed to bring back the ability of the Error Reporting Server to follow the status of report-linked Github issues and update the same for the reports that it is linked with. Github provides a interface to listen to events through webhooks. Webhooks allow developers to listen to a variety of events like push, issues etc. I added a controller in the Error Reporting Server which listens to all the issues events and if it receives an event for an issue which is linked to by any of the reports, it updates the status of the linked reports accordingly.
For the next week, I would be taking on two tasks #159 and #160, which were not originally part of the proposal, but seem to be natural extensions to the task in #98.
Thanks :) I have faced some issues while adding left and right scrolling icon and its functionality but with little more debugging I have made it possible.
Next while working on table structure page when I have written this in js/menu-resizer.js the icons are loading n number of times where n is number of columns in the table. It was working fine on all pages except table structure page.
Last issue that I faced was while adding scrolling inside the fieldset tag seems like the width in percentage was not working so I used javascript here to implement this.
Rest all implementation were straight forward. Just needed some debugging on browser developer tools.
Also I have made some standards for how I am naming the new classes the details of which I will add after completing it because I think more will come as I keep working on it.
Weblate 2.15 is almost ready (I expect no further code changes), so it's really great time to contribute to it's translations! Weblate 2.15 should be released early next week.
I'd especially like to see improvements in the Italian translation which was one of the first in Weblate beginnings, but hasn't received much love in past years.
It has been few months since I'm providing Windows binaries for Gammu, but other parts of the family were still missing. Today, I'm adding python-gammu.
Unlike previous attempts which used crosscompilation on Linux using Wine, this is also based on AppVeyor. Still I don't have to touch Windows to do that, what is nice :-). This has been introducted in python-gammu 2.9 and depend on Gammu 1.38.4.
What is good on this is that pip install python-gammu should now work with binary packages if you're using Python 3.5 or 3.6.
Maybe I'll find time to look at option providing Wammu as well, but it's more tricky there as it doesn't support Python 3, while the python-gammu for Windows can currently only be built for Python 3.5 and 3.6 (due to MSVC dependencies of older Python versions).
Hosted Weblate provides also free hosting for free software projects. The hosting requests queue was over one month long, so it's time to process it and include new project.
If you want to support this effort, please donate to Weblate, especially recurring donations are welcome to make this service alive. You can do them on Liberapay or Bountysource.
As mentioned in my previous post, I am working on making new UI for multi-table query for phpMyAdmin. I believe I can safely say it is in good shape as of now, although code review is still pending. I have added most of the features from earlier interfaces. Some of the examples of queries which you can construct are:
Basic queries.
select `wp_posts`.`*` from `wp_posts`;
select `wp_posts`.`ID` from `wp_posts`;
Aliases.
select `a`.`ID` from `wp_posts` as `a`;
select `a`.`ID` as `id_col` from `wp_posts` as `a`;
WHERE clause
select `wp_posts`.`ID`, `wp_posts`.`post_title` from `wp_posts` WHERE `wp_posts`.`ID` > 49 AND `wp_posts`.`ID` < 74;
ORDER BY clause
select `wp_posts`.`ID`, `wp_posts`.`post_title` from `wp_posts` WHERE `wp_posts`.`ID` > 49 AND `wp_posts`.`ID` < 7 ORDER BY `wp_posts`.`post_date`;
Following is a screenshot of the UI:
I had a plan to add filtering and pagination of queries, but luckily they are built-in for query response HTML. You can checkout code from the PR and try it yourself. As always, any feedback and suggestions are more than welcome.
According to my timeline, I am supposed to complete this with code-review by 15th June. But I understand that other developers may be busy right now. So, I will be moving on to my next task this week i.e. Default options for transformations.
I will update the code for Multi-Table querying once I get code reviews.
![[RSS 1.0 Feed]](images/rss10.png){kind=small}
![[RSS 2.0 Feed]](images/rss20.png){kind=small}
![[Atom Feed]](images/atom.png){kind=small}
![[FOAF Subscriptions]](images/foaf.png){kind=small}
![[OPML Subscriptions]](images/opml.png){kind=small}
![[Venus]](images/venus.png){kind=small}
