Page 1 of 1

[python, mysql, web] one big file OR many small files

Posted: Tue Dec 25, 2012 1:55 am
by Isaac
I'm in a strange situation where a server admin of a friend won't allow us to use the mysql python library, claiming it's a security risk. So while he's shopping for a new webhost I was thinking of a temporary work-around.

Currently we're saving user posts/comments to text files. But after some thinking I realized that I could just as easily save one post per file. This would eliminate the problem of accidently overwriting another users comment. It might even save space, compared to using a single big text file, since only comment data will be put into the text file. Note that the post ID would now be the file name and the timestamp might just be drawn from when the file was created.

I think that using multiple files will work better than a single file and plan to make the modifications this week. So far, I can't think of any draw-backs I could have, but I'm guessing there might be some obvious things I'm missing. Please feel free to point out why this is a bad idea.

Re: one big file OR many small files

Posted: Tue Dec 25, 2012 4:53 am
by Jeff250
I would just try to expedite "he's shopping for a new webhost" and then write for MySQL.

You can set up your own development environment inside a virtual machine in the meantime.

Re: one big file OR many small files

Posted: Wed Dec 26, 2012 11:43 am
by Isaac
Hmm.. question:

Setting up a virtual machine sounds great, except for the part that using home internet. Home Internet would probably be slower and cause more problems than simply saving to text files, no? I'll try it anyway, but I don't like it.

Re: one big file OR many small files

Posted: Wed Dec 26, 2012 3:43 pm
by Jeff250
Development environments are usually slower than production environments in a number of ways, but people still use them because 1) you typically don't notice the slowness if there are only one or two developers poking at the server at a time and 2) even if noticeable, it's not a problem as long as the slowness neither hinders development nor will exist in production.

You're obviously concerned about the amount of time things take, so I'd focus on the amount of time it takes to rewrite your code for a small performance improvement that will only be relevant for an increasingly irrelevant situation. Optimistically, your rewrite will shave off a few milliseconds from request times until you move to a new server where you can use MySQL, but your application's development will be delayed by the amount of time that it takes to complete that rewrite.