Fatdog64 updates - behind the scenes

Fatdog64 was last released on 23 Jan. And although we have been quiet, it doesn't mean that things stopped. Things are still going in the background, bug fixing, adding features, etc.

Here are a few major things that has been going on, in no particular order.

1. Puppy Linux forum user ICPUG has indicated possible issues with Fatdog savefile residing on an NTFS partition, especially partition shared with Windows. This is due to the way Fatdog uses ntfs-3g: it runs with full permission control enabled. It makes NTFS behaves like POSIX filesystem and we can use it for save directory (not only save file), but on the other hand it makes Windows complain about not-granted access each time the partition needs to be accessed from Windows.

Based on this feedback, we have added "ntfsnoperm" boot parameter to disable that permission control. When this parameter is used, ntfs-3g is run without permission: it would behave just like FAT filesystem (all files and directories are owned by a given uid/gid specified at mount time). It would not be possible to use a save directory on NTFS, but at least the permission wouldn't be touched and Windows will stop complaining.

2. The same ICPUG also found an old bug related to the above: where the user "spot" would be unable to access a Downloads folder that has been relocated to a save partition if the save partition is NTFS. This has been fixed.

3. Fatdog has long supported btrfs. The kernel is built to support btrfs and we have the complete btrfs-progs included; and nominally we support having save directory on btrfs. But this does not actually work; because "aufs" - the unifying filesystem layer that we use to make the magic happen, does not support btrfs the way it supports other filesystem. For the techie: the problem is that aufs cannot have its xino file inside btrfs. It has to be somewhere else.

Thanks for our team member SFR (his Puppy Forum name), this was brought to attention and he shared a fix too. The fix was tested, worked on, and finally merged: now, save directory will work seamlessly on btrfs-formatted partition.

4. The original motivation for doing (3) above was actually to attempt to use a compressed filesystem. This has been a particular interest for me for years. The last time I explored this (a couple of years ago), btrfs wasn't mature enough and there was no other native writable filesystem with compression support. Sure, there were native filesystem with compression support - but they were all readonly (squashfs being the most popular one). Sure, a couple of FUSE filesystems support compression too (in fact it's their number one feature), but they were not actively maintained and being done in userspace, it was slow. I never got enough motivation to implement them properly with Fatdog.

Btrfs finally changed this equation. By now it should be considered mature enough (although probably not as hyped as before), and it supports compression as well. In fact, it supports three different compression algorithms: lzo, lz4 and zstd.

So as we fixed btrfs save directory support, we grafted compression support too. If the kernel parameter "btrfscompress" is specified, the compression support for btrfs will be enabled, using the algorithm specified on that parameter. This has been tested to work wonderfully and using zstd, the compression rate is about 30% on average.

5. We have an update on the in-house "screencaster" application (a program record the video of the display). It now has the ability to take repeated screen capture, better quality video, among other things.

6. Step, another member of Fatdog team, has also revamped the "Samba Shares" application. It has been re-factored and is heavily tested to work across different Samba servers (different Windows versions, etc).

7. We have also fixed a long-standing bug when running Fatdog with RAM layer operation, which can cause inconsistencies if the system is heavily used (a lot of filesystem access) when the "merge down" process happens. SFR found this to be irritating enough to find a solution that works better; and this got incorporated into Fatdog.

An additional feature was also added: the ability to remove whiteouts if they don't hide any files on the lower SFS layers, but not activated by default due to possible conflict with multisession operation (we haven't tested the interaction yet).

8. We also include the "updater script" which re-enable VLC's ability to play youtube video directly, without having to update VLC itself. The script is called update-vlc-playlist-luac.sh. SFR found this script.

And of course, many other smaller bugfixes and feature addition, as well as adding and updating packages on the repo as well.

So when will we have a new release? Well, it will be released when it is released.


Posted on 19 Apr 2020, 16:17 - Categories: Fatdog64 Linux
No comments - Edit - Delete


The road to hell is paved with good intentions

Good intentions, good deeds, now matter how good they are, when carried to its logical conclusion, always lead to destruction.

Because when we go north to reach the most extreme northern point, without knowing what the north pole actually is, we will never reach it.

We need directions. We need sign posts. And fortunately there are sign posts and directions to north pole, if that's all we want to go.

How about sign posts of life?

There is this someone. He called himself as The Way. The Way that will lead you to the Truth. The Truth about Life, and beyond. The Truth that will set you free.

But He, like all sign posts, doesn't demand that He be followed.
It is up to us if we want to.





Posted on 11 Apr 2020, 18:09 - Categories: General
No comments - Edit - Delete


Nothing to Fear, but Fear Itself

Thus said Franklin D. Roosevelt, the 32nd president of the United States, even though I'm sure many wise men before him have said the same, although, perhaps, not in the same exact words.



Fear of something is almost always worse than that something itself.

Posted on 5 Mar 2020, 02:14 - Categories: General
No comments - Edit - Delete


Love at first sight

"Love at first sight often ends at the first quarrel."
---
The Book of Ten Children, 1:2


Posted on 21 Feb 2020, 16:37 - Categories: General
No comments - Edit - Delete


Fatdog64 810 Final is released

Maintenance release, basically a bug-fixed version of 810 Beta.

Release Notes and Forum announcements

Can't believe that Fatdog is coming to its twelfth year.



Download locations: ibiblio.org, aarnet, uoc.gr, nluug.nl.

Posted on 23 Jan 2020, 21:53 - Categories: Linux Fatdog64
No comments - Edit - Delete


CSVfix patches for regex and exec

CSVfix is a tool for manipulating CSV files. Along with the usual column re-ordering and filtering, CSVfix offers a powerful per-cell data transformation using a simple expression language, as well as regular-expression for string matching and editing. And if this is not enough, CSVfix can execute external process - for every cell that needs to be processed. And oh, it's available for Windows too!

I find this tool to be very handy in what I need to do, so when I encountered a bug in its regex processing (for its "edit" command), I immediately checked if there is any updates to this tool. Unfortunately, its development seems to have ceased in 2015; and no other people seem to have picked up the development (I did find some forks, but they were all older copies from when it was still hosted in google code).

So I set out to figure out about the problem and hopefully rectify it. I found that the problem was in its regex library, which was a home-grown library (apparently adapted from an algorithm book). It is 2020 as of this time of writing, and C++ now comes with its own STL regex library (std::regex). I decided to rip off the custom regex lib and replace it with the STL regex instead, while keeping the rest of the class interface identical, therefore no other part of the code needed to be changed. This instantly fixed the problem, and as a bonus, now we can use ECMAScript-compatible regex instead of just the basic regex.

Later, I found out that the "exec" command also had a bug (the flag "-ix" did not work properly), so I traced this and fixed it too.

Oh, and during the process, I tried to run its testsuite - and while most of them passed, some did fail. Mainly because of CRLF/LF inconsistencies, so I changed those the test data to use LF. It is also a warning that this tool only works with platform "newline" - CRLF in Windows, and LF in Linux - so if files were to be exchanged between platforms, they must be properly translated first before use.

Here are the individual patches.
- regex patch
- exec patch
- test-case patch

They apply on top of the commit 93804d4 from 2015-02, which was the latest when I wrote this. They are licensed in the same way as the original CSVfix is licensed.




If CSVfix is not powerful enough for you, there are other similar tools:

1. miller is a tool in very similar spirit with CSVfix, but it is (much) more sophisticated. Its "data transformation language" looks more expressive than the one in CSVfix. If you have a problem you cannot solve with CSVfix, miller will probably help you. As a bonus, it is still in active development - that means bugs will be squashed. It is written in C, you will need to compile it if it is not in your package repository. (Fatdog, naturally, has it in its repository).

2. csvkit is a collection of tools that more or less perform the same functions as CSVfix. It supports direct conversion to/from Excel files, importing/exporting into databases (sqlite and postgresql as documented, perhaps others too), as well as running direct SQL queries from CSV files (and databases too). It is written in Python3 so you can install it using pip3. Fatdog has this in its repository too (so you can install it using package manager instead of pip3).

3. rbql basically enables you to run SQL-like on CSV files; but its power is its ability to run python (or javascript, depending on which backend you choose) code for every cell. Fatdog has it in its repository too, although you can just use pip to install it, if you don't run Fatdog.

Posted on 23 Jan 2020, 21:53 - Categories: General Linux
No comments - Edit - Delete


The Good Book

Everyone keep two books. The "good book", and the "bad book". People that we know, we like, and generally we like to be associated with, is in our "good book". People that have hurt us, or people who behave in a manner that we despise, on the other hand, get their names written in our "bad book".

Being listed in either book have consequences, both good and bad.

Now, God carries two books too. Would you rather be in men's good books, or God's good book? How do you live your life - are you striving to be in God's good book, or to be in other men's good books?


Posted on 4 Jan 2020, 02:05 - Categories: General
No comments - Edit - Delete


Welcome to the new decade ... or not!

This new year 2020 is a new year that is divisble by ten. It changes the second digit from "1" in 2019 to "2" in 2020.

A lot of people are wishing me happy new year and also welcoming me to the new decade. I am grateful for their well-wishes.

Except for one thing.

2020 is not the start of the new decade. Just like year 2000 is not the start of the new millenium. The new decade starts at year 2021. Just like the 21st century starts at 2001, not at 2000. Year 2000 belongs to the 20th century.

In case you don't see why: it is because we start counting our calender at Year 1 AD. So the first decade - all ten years of it - would be the counting numbers from 1 to 10. The second decade starts at year 11 AD.

But of course, I'm being pedantic. Who cares anyway. Calendars have always been subject to various "adjustments"; and different cultures use different calendars. Those still using Julian calendars have not celebrated the new year yet, it is still 9 days away at the time of writing.

PS: We do not have Year Zero. The year before 1 AD is year 1 BC.

Posted on 4 Jan 2020, 01:42 - Categories: General
No comments - Edit - Delete


Fatdog64 810 Beta is Released

Maintenance release.

Release Notes and Forum announcements



Download locations: ibiblio.org, aarnet, uoc.gr, nluug.nl.

Posted on 9 Dec 2019, 02:37 - Categories: Fatdog64
No comments - Edit - Delete


The Author of Life

I am an author. I write both fiction and non-fiction works.

When I write fiction stories, I make and write my own characters. These characters are people who live in my fictional world, fictional universe, oblivious to the fact that they are all fictitious. For them, everything that happens in their world is as real as it could be.

Now, of course, those characters I have created: they are not independent of me. They exist because of me, they exist in me. Eventually, I do get to decide that they see, that they hear, what they feel, what they do. I determine their fate, their destiny. I set how life is going to be for them, and what revolves in their world.

But they are not just characters, or puppets. In my mind, they are alive, and I'm only writing part of their life that I happen to see. Not only that, I love my characters, and I care about what happens to them. Certain characters are lovable, and some are detestable - but I certainly care about all of them.

I wish - if it were ever possible - to actually meet my characters, in their own universe. See how they live, how they feel. Feel their joy, and suffer their sadness. Be one of them. And tell them, how much I care and love all of them. That all of their life has meaning to me, their author.

---

If you are an author, or an artist of any kind that loves your own created arts, it is easy to see that you, too, is a work of art of an omnipotent Author, that is, God.

When God paints, you see the flowers, country side, and the star constellations.
When God sculpts, you see the mountains and the Laniakea supercluster.
When God builds, you see the quarks self-assembling to atoms and the visible universe.
When God engineers, you see protein machines and solar systems and life.

And when God writes, you see yourself.

You exist because of Him, your Author.

He cares about you, more than you care about your own creation.

He loves you so much that He gives you life, and more: something that you can never give to your own creation: a free and independent mind to decide what you want to do about Him - whether to love Him back, or to reject and deny Him.

He loves you so much that He came into this world, to feel its joy and suffering, and to tell you that He love you, and to tell you that your life is meaningful, and that there are more to your life than just this world; He showed the way to Himself.

I thank my Author for giving me life.
I thank my Author for loving me more than I love myself.
I thank you, Lord Jesus, and I long to see you face-to-face when the time comes.

Posted on 30 Jul 2019, 13:43 - Categories: General
No comments - Edit - Delete


Pages: [1] [2] [3] [4] [5] ...