English Lua database system (CDB)?

17 replies
Goto Page
To the start Previous 1 Next To the start
Poll Poll
will u use CDB database system for saving data ?
Only registered users are allowed to vote
yes its cool xD
60.00% (9)
no, i don't care :/
40.00% (6)
15 votes cast
26.06.17 09:52:11 pm
Up
_3yrus
User
Offline Off
IMG:https://docs.microsoft.com/en-us/media/hubs/sql/sql-server-1.svg


Hi there !

i started writing a database system for lua to make data storing easy !

in my structure folders are database and files are tables !

and my question , would it be useful for you ?
if yes so i start developing main part of it ,if not state your ideas about this !

thanks for reply in advance !

INFO This only works on windows os !

the code and the usage can be found below : >
edited 9×, last 03.07.17 01:42:14 pm
here we go !
26.06.17 10:04:24 pm
Up
Starkkz
Moderator
Offline Off
I would be up to work with you if it was a relational database with the DDL/DML SQL standard.

Not to mention that the code must be object oriented for a better aproach.
lol
26.06.17 11:21:21 pm
Up
_Yank
User
Offline Off
Seems cool however I can't think of any situation where I'd need to use it. I ask you the same question, will it be useful for you ? What will you be using it for ?

I also recommend you elaborating the thread, for example, you could explain how the database system works for people who haven't heard of this term before.
28.06.17 08:05:58 am
Up
_3yrus
User
Offline Off
this post is trash
edited 13×, last 03.07.17 06:35:33 am
here we go !
30.06.17 04:34:21 pm
Up
Infinite Rain
Reviewer
Offline Off
Why would I use this over something like SQLite?
A thousand may fall at your side, ten thousand at your right hand, but it will not come near you. You will only look with your eyes and see the recompense of the wicked. - Psalm 91:7-8 ESV
30.06.17 04:50:23 pm
Up
_3yrus
User
Offline Off
@user Infinite Rain: bcs its very fast and in big (and also in small) reduce code size and when you need to get information u can do many incredible things . (What is SQL)
for example : regular way for getting saved players data :
1 - use io to open txt file
2 - analyze the text and then use it
e.g :
Code:
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
local filename = "sys/lua/saves/%s.txt"
          local file = io.open(filename:format(player(id,"usgn"), "r"))
          local line
          if not file then
               line = {0, 1}
          else
               line = file:read("*a"):split()
          end
          score[id] = tonumber(line[1]) or 0 -- If line[1] is not a number, level[id] becomes 1
          vip[id] = tonumber(line[2]) or 0 -- Same as above reasoning (prevents errors)
     else
          msg2(id,"©255000000No USGN found!@C")
          score[id]=0
          vip[id]=0
     end

the example is just a simple on !

and for storing players data u need code more than it .

but in new system :
for load data :
Code:
1
CDB.query("SELECT score,vip FROM players")

and for save :
Code:
1
CDB.query(""UPDATE players SET score = "..sc..", vip = "..vip.." WHERE usgn = "..usgn)


the second code exactly do the same , but very very very .. easier !

and also the data you have stored is very arranged !
edited 3×, last 30.06.17 05:22:33 pm
here we go !
01.07.17 09:24:59 am
Up
MikuAuahDark
User
Offline Off
@user Infinite Rain: Because you'll have to compile SQLite3 Lua bindings & unfortunately Lua in CS2D is statically linked, so it's possible that you'll getting memory-related errors.

@user _3yrus: From your example, is there a way to use prepared statement?
file cs2d LuaJIT for Dedicated Server (9) JIT POWER! | Know your Lua errors! | don't use PM for scripting help!
01.07.17 10:16:02 am
Up
Infinite Rain
Reviewer
Offline Off
How is compiling something a problem given that the result would be much better than anything offered by the OP? And besides, getting the compiled version with LuaRocks is very easy.
A thousand may fall at your side, ten thousand at your right hand, but it will not come near you. You will only look with your eyes and see the recompense of the wicked. - Psalm 91:7-8 ESV
02.07.17 11:16:21 am
Up
_3yrus
User
Offline Off
user MikuAuahDark has written:
@user Infinite Rain: Because you'll have to compile SQLite3 Lua bindings & unfortunately Lua in CS2D is statically linked, so it's possible that you'll getting memory-related errors.

@user _3yrus: From your example, is there a way to use prepared statement?


as the result of the poll i think i have to stop project bcs i think its a useless effort
edited 2×, last 03.07.17 06:41:12 am
here we go !
02.07.17 11:56:09 am
Up
Gaios
Security Supporter
Offline Off
I'm not sure if you're enought good to write such framework, becaues of your l33t language. For me you're guy who want to do something cool without knowledge.
02.07.17 04:39:04 pm
Up
_3yrus
User
Offline Off
@user Gaios:
U jerked me
So what would u do if wrote it?
here we go !
02.07.17 07:41:22 pm
Up
Avo
User
Offline Off
@user _3yrus: Oh, don't listen to him. Just think if it's worth doing something that won't be used by the community really often.

If you feel good at lua, maybe you should think about helping with thread cs2d New Bot System - Testing Version Released or thread cs2d Learning / Evolving Bots | Machine Learning. New and more effective bots are really needed since CS2D is going to be published on Steam.
02.07.17 08:52:34 pm
Up
_Yank
User
Offline Off
@user _3yrus: Don't give up on the project! It looks promising and I believe it certainly has the potential to become useful.

You definetely should try to improve it listening to the suggestions people make. Think also of features and procedures that would make it a better choice over other options such as SQLite as user Infinite Rain said.

For example, I think that you should start by considering Starkkz comment about following DDL/DML SQL standards.
03.07.17 06:24:27 am
Up
_3yrus
User
Offline Off
thanks guys,
i would finish this project anyway, whether its useful or not ! (and i think it would be).

just pray for me i have university entrance exam the day after tomorrow, and i'm really anxious , pray for me
and after the exam i will start project and finish it with ur cooperation !

and about DDL/DML SQL standards : its exactly would follow DDL/DML SQL standards.(i have experience working with it)


New Syntax :

More >


And new code :

Code part 1 >


Code part 2 >
edited 2×, last 03.07.17 06:40:30 am
here we go !
03.07.17 05:45:13 pm
Up
_Yank
User
Offline Off
The project itself is not bad at all but I believe that its code and its usage can certainly be improved. Here's a few more suggestions:

- An Object Oriented approach (http://lua-users.org/wiki/ObjectOrientationClosureApproach)
- Relational Structure
- Implementation of the most useful SELECT variants (like, DISTINCT, COUNT, TOP, FIRST, IN, etc)

I'm sure that if these features were present, you would get an actual approval of most of the guys here.

Also some quick notes that can be easily done:

- Make a dedicated log function (so you don't have to be always typing msg('CDB:...'))
- Replace the © with \169 on that log function (so you have no problems with encoding)
- Upload the script somewhere as a .lua (so we can read it properly)
- Avoid unnecessary functions/values (that you only use once and similar)
- Avoid creating functions with common names under _G (file_exists and similar, I'm look at you). Put them under their own table (for example, CDB.core.file_exists) or give them a prefix (CDB_fileExists is another example).
- Separate the example script and the actual library, it is not a good idea to keep them on the same file

Here's a good example of a database managing script I would use:
Code:
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
cdb = require "sys/lua/cooldatabasescript"

DBMS = cdb.install("sys/lua/myScript/databases")
A = DBMS.database_create("males")
B = DBMS.database_create("cooler_males")
C = DBMS.database_create("females")

B:rename("dominant_ones")
C:drop()

if not DBMS.database_exists("females") then
     print "The inferior species have been successfully wiped out"
end

-- Note that I'm not specifing the types (to simplify things)
A:table_create("KoolKids","name:Player,age:0,endurance:100,speed:100,swag:100")
A:table_rename("KoolKids","_Yank's Disciples")
A:table_move("_Yank's Disciples", "cooler_males")

if not A:table_exists("KoolerKids") then
     print "Yeah, KoolKids still the dankest"
end

-- Bla bla bla, moar code here..

-- Then, the general DBMS functions (query and execute for example) would work under the last touched database
DBMS.execute("UPDATE KoolKids SET swag = 0 WHERE name = Starkkz")
-- But, just in case, you could also...
A:select()
-- or
A:execute("UPDATE KoolKids SET swag = 0 WHERE name = Starkkz")

-- Just a simple joke illustration
edited 1×, last 03.07.17 08:17:25 pm
03.07.17 06:14:13 pm
Up
_3yrus
User
Offline Off
@user _Yank: really thanks for guidance , i would implement them asap.

----- new ------

hi guys ,i'm done whit my exam and i think i was great

so i started project again , first thing i did is make code object ordinated as yank said ->

new codes Object Ordinated :

part 1 >

part 2 >


Code:
1
2
3
4
5
6
7
-- example

CDB.install()
con = CDB:connect('mydb') -- or con = CDB.create_database('mydb')
con:create_table('table1','some data')
local results  = con:query("SELECT ...")
con:close()


and now its ready for next steps.
edited 4×, last 07.07.17 05:05:37 pm
here we go !
08.07.17 12:03:52 am
Up
VaiN
User
Offline Off
I'm not trying to be negative or discourage you. I just feel the need to provide some feedback regarding this concept.

TL;DR: This concept has all of the downsides of SQL without the upsides.

I'm a bit confused why you'd want to create a system like this just to make simple data management more convoluted. The main benefit of a database is being able to perform queries over a large amount of data very quickly. My Rewards server had over 10,000 user data files at one point, and so I know how handy database support would be. With using tables as files in a file system, SQL just isn't practical at all. Imagine something like:

Code:
1
SELECT * FROM db WHERE kills > 100


To do this, it would have to go through every file, and if you have a lot of them, it would slow the server to a crawl. The whole concept of the WHERE clause is flawed with a file system approach because the only efficient use would be the name of the file itself. Otherwise, again, it would have to check every file/table for a match. And if you're only working with a single file at a time anyway, then it's not really doing anything that would require a SQL query.

Here's how I look at this. I'm approaching this as being used for managing player data (stats, ranks, levels, etc), as I'm not sure what else it would be for. When hosting a server, you have a max of 32 players at once. There's no reason to not have the data of each player loaded in memory while they are in-game, as it really doesn't use much. There's no added benefit of loading, updating, and saving a single value in the player's data file versus just re-saving the entire table from memory. It's also faster to use loadfile() to get all of that player's data at once, than to read a file line by line and parse it for a value.

So if you're loading all of the file at once, you might as well just leave the table loaded in memory. Then it's just a matter of "data[id].whatever" rather than typing out a whole select statement. If, however, you are trying to work with all data of all players loaded in memory, imagine how that would be with my old server having over 10k entries. Needless to say, that would be a bit much.

And so, the only reason I can see why you'd want to use this system, is just for the SQL syntax. That seems counter-productive to me, because it's a lot easier to make mistakes that are harder to catch when you are passing around long strings as logic, instead of code. On top of that, you have the added complexity of parsing the logic in the queries. The only benefit I can see is that it'd allow doing something in a single function instead of two. Seems like a lot of extra work, with very little potential for any gain and a lot of potential for user error.

Again, my understanding of this is just based around managing player-specific data, or similar use case. If you are intending to use this some other way, then it might help us understand better if you elaborate on what scenarios where SQL would be more beneficial than Lua within the constraints of CS2D scripting.

BTW, if you are just saving data as Lua tables, check out SALT.
If you are wanting a database, I'd imagine you are working with a lot of data, and that's what SALT is designed for. If nothing else, it could be a good backend to the SQL-like frontend.

Another tip, your code is Windows-only because of your use of os.execute() to do file management. You should try to keep things as OS-agnostic as possible, such as deleting and re-creating a file rather than using OS-specific commands for moving/renaming them. This all depends on your use case of course. Making a system that is Windows-only would make it far less likely to be adopted, as the majority of servers are run on a Linux VPS, or at least the ones that would need such a system.

Regarding database support in general:
I would love to see real database support in the game, as it would allow a lot more advanced methods of managing data, getting top-ten lists, preventing duplicate users, ect. But with how Lua is currently embeded in CS2D, there's no way to use external database libraries reliably. There could also be performance issues as there's no threading either. It's too bad LuaJIT support was rolled back, as that could have been amazing. But you don't have to do everything inside the game. Externally, Lua can work with databases very easily, and with a bit of inter-op creativity, you can still get those top-ten lists in-game.

Sorry for the long post.
08.07.17 01:00:26 pm
Up
_3yrus
User
Offline Off
@user VaiN: tnx for tips !
here we go !
To the start Previous 1 Next To the start