[python, mysql, web] one big file OR many small files

For all coding issues - MODers and programmers, HTML and more.

Moderators: Jeff250, fliptw

Post Reply
User avatar
Isaac
DBB Artist
DBB Artist
Posts: 7127
Joined: Mon Aug 01, 2005 8:47 am
Location: Ơ̸̦͇̲̬̭̱̰͎̞͈̣͎͚̳ͬ͋̃̀̇͊͂͋͐ͦ̽ͣ̂ͥ͊̅̀̚͠ B̶͖̯͉̜̰̲̓̔͋̈́ͅ È̯ Y̪̤̼͉̠̙͝

[python, mysql, web] one big file OR many small files

Post by Isaac » Tue Dec 25, 2012 1:55 am

I'm in a strange situation where a server admin of a friend won't allow us to use the mysql python library, claiming it's a security risk. So while he's shopping for a new webhost I was thinking of a temporary work-around.

Currently we're saving user posts/comments to text files. But after some thinking I realized that I could just as easily save one post per file. This would eliminate the problem of accidently overwriting another users comment. It might even save space, compared to using a single big text file, since only comment data will be put into the text file. Note that the post ID would now be the file name and the timestamp might just be drawn from when the file was created.

I think that using multiple files will work better than a single file and plan to make the modifications this week. So far, I can't think of any draw-backs I could have, but I'm guessing there might be some obvious things I'm missing. Please feel free to point out why this is a bad idea.
s☼-£♦и̫͍ͥ̍ͪ͌̓͗͡о̡̹̱͊̅ͮ̓̕͢б̧̝̻̪̤̳̜͐̓̉ͤ͢͜ ͙̬͙̆̑ͮ̐ͭ̾̂́͘i̎̌̾̓̽̀̈̓̀҉͉̙̦͎̘̝͕f̻͕͔̘ͣͣ̓͊̿͢͜ ͍͔͈͕̮̫ͣ̆ͮ̊͋/♂6Æ!♪╩"▲L└уͭ̂͐̇҉̴̣̼̞̠̯͓̺̞ф̜̊͌̈́̋̏̐́ц̨͔̮̿̇ ̨̛͖̙͖̖̮̗̱ͩ̆͞ͅа̥͇̞̖͚̟̅͐ͤ͞͠͠э̜̘̩̳̬͔̾ͯ̀ͫ̒̐̿ͅͅг̭̖̀ͦ̒̑ͥ̌ͮͫ͞ё͔̟̃ͬ̾̓͟ё̦̞̙̫͔̩͑̀͂ͯ̄̔̃̑̀͠ͅͅ
User avatar
Jeff250
DBB Master
DBB Master
Posts: 6386
Joined: Sun Sep 05, 1999 2:01 am
Location: ☃☃☃

Re: one big file OR many small files

Post by Jeff250 » Tue Dec 25, 2012 4:53 am

I would just try to expedite "he's shopping for a new webhost" and then write for MySQL.

You can set up your own development environment inside a virtual machine in the meantime.
User avatar
Isaac
DBB Artist
DBB Artist
Posts: 7127
Joined: Mon Aug 01, 2005 8:47 am
Location: Ơ̸̦͇̲̬̭̱̰͎̞͈̣͎͚̳ͬ͋̃̀̇͊͂͋͐ͦ̽ͣ̂ͥ͊̅̀̚͠ B̶͖̯͉̜̰̲̓̔͋̈́ͅ È̯ Y̪̤̼͉̠̙͝

Re: one big file OR many small files

Post by Isaac » Wed Dec 26, 2012 11:43 am

Hmm.. question:

Setting up a virtual machine sounds great, except for the part that using home internet. Home Internet would probably be slower and cause more problems than simply saving to text files, no? I'll try it anyway, but I don't like it.
s☼-£♦и̫͍ͥ̍ͪ͌̓͗͡о̡̹̱͊̅ͮ̓̕͢б̧̝̻̪̤̳̜͐̓̉ͤ͢͜ ͙̬͙̆̑ͮ̐ͭ̾̂́͘i̎̌̾̓̽̀̈̓̀҉͉̙̦͎̘̝͕f̻͕͔̘ͣͣ̓͊̿͢͜ ͍͔͈͕̮̫ͣ̆ͮ̊͋/♂6Æ!♪╩"▲L└уͭ̂͐̇҉̴̣̼̞̠̯͓̺̞ф̜̊͌̈́̋̏̐́ц̨͔̮̿̇ ̨̛͖̙͖̖̮̗̱ͩ̆͞ͅа̥͇̞̖͚̟̅͐ͤ͞͠͠э̜̘̩̳̬͔̾ͯ̀ͫ̒̐̿ͅͅг̭̖̀ͦ̒̑ͥ̌ͮͫ͞ё͔̟̃ͬ̾̓͟ё̦̞̙̫͔̩͑̀͂ͯ̄̔̃̑̀͠ͅͅ
User avatar
Jeff250
DBB Master
DBB Master
Posts: 6386
Joined: Sun Sep 05, 1999 2:01 am
Location: ☃☃☃

Re: one big file OR many small files

Post by Jeff250 » Wed Dec 26, 2012 3:43 pm

Development environments are usually slower than production environments in a number of ways, but people still use them because 1) you typically don't notice the slowness if there are only one or two developers poking at the server at a time and 2) even if noticeable, it's not a problem as long as the slowness neither hinders development nor will exist in production.

You're obviously concerned about the amount of time things take, so I'd focus on the amount of time it takes to rewrite your code for a small performance improvement that will only be relevant for an increasingly irrelevant situation. Optimistically, your rewrite will shave off a few milliseconds from request times until you move to a new server where you can use MySQL, but your application's development will be delayed by the amount of time that it takes to complete that rewrite.
Post Reply