How do I make an Autocleric?

Posted by Trigger on Tue 27 Feb 2007 10:00 PM — 38 posts, 133,052 views.

#0
Ive been trying various ways to make this trigger but I really cant quiete understand your trigger system. I love your Mushclient and am enjoying it almost every day but I cant make this autocleric trigger.

If the mud sends this string:

Destroy says, 'heal'.

I want it to automatically

cast heal Destroy

For some reason this trigger

(.*) says, '(.*)'.
cast %2 %1

sends
cast heal'. You say, 'Pallando


I was hoping you guys would help me with this subject cause Im not to good at reading the helps files on the matter.
Trigger
Amended on Tue 27 Feb 2007 10:24 PM by Trigger
Australia Forum Administrator #1
I don't see why it would do that. Can you copy and paste the actual trigger, see this post for how to do that:

http://www.gammon.com.au/forum/bbshowpost.php?bbsubject_id=4776
USA #2
Just a quick question to Trigger. Are you testing this out by sending "say Pallando says 'heal'." to the mud? If so, the trigger would capture "You say, 'Pallando" as the first wildcard, and "heal'." as the second wildcard.

If this is the case, try testing it out with "emote says 'heal'." or open up the simulated output window with ctl-shift-F12, and put in "Pallando says, 'heal'" in the text area.

Also, keep in mind that what you wrote is an easily exploitable bot, and if botting is illegal in whatever mud you're playing on, you will get caught easily with that trigger. might be better to match on something like this:
^(/w+) says, '(heal|cure blindness|etc.)'.$
Still a bit exploitable, and still botting, but much more controlled. Otherwise, you might have people like me following you around going "say ventriloquate <yourname> I'm botting, and it has been proven by"
Amended on Wed 28 Feb 2007 01:50 AM by Shaun Biggs
#3
<triggers>
<trigger
enabled="y"
match="* says, '*'."
sequence="100"
>
<send>cast %2 %1</send>
</trigger>
</triggers>


Is the trigger copied
and when I use this one

<triggers>
<trigger
enabled="y"
match="(.*) says, '(.*)'"
regexp="y"
sequence="100"
>
<send>cast %2 %1</send>
</trigger>
</triggers>


The same thing happens
Amended on Wed 28 Feb 2007 03:54 AM by Trigger
USA #4
I'd investigate what Shaun suggested. How are you testing the trigger?

Also I'd suggest you make sure that botting is allowed wherever you're playing. Some places consider it to be ok. On my MUD it is considered a very bad form of cheating.
Australia Forum Administrator #5
I agree with the previous posters. You said that the triggers sends " You say," but those words appear nowhere in the trigger, so they must be in the MUD output.

In fact, the whole idea needs to be rethought a bit. The way you have written your trigger it will match on all says. So, for example, if you get this line from the MUD:


Nick says, 'Hi there, let's go meet at the inn'.


Then your trigger will match, and you will send:


cast Hi there, let's go meet at the inn Nick


You probably need to do what Shaun said, and restrict it to certain words like "heal" and so on.
#6
It works great but Im still having trouble.
I want to cast

Remove poison

When someone says

Rem
or even
Remov
removepo


are anything after
rem
what should I do
what did I do wrong

<triggers>
<trigger
enabled="y"
match="(.*) tells you, '(Rem+ovePoison|Rem+ovepoison|rem+ovePoison|rem+ovepoison)'."
regexp="y"
sequence="100"
>
<send>cast %2 %1</send>
</trigger>
</triggers>


Also while I have your attention
Is there a way to
ignore the fact that it is capital
so that if they type
RemovePoison
or
Removepoison
it casts without having to make the trigger
(RemovePoison|Removepoison|removePoison|removepoison)
?

You Guys are great
Amended on Wed 28 Feb 2007 04:47 PM by Trigger
USA #7
First, the easy question to answer. There is an "Ignore case" checkbox in the trigger wizard. That will match "heal" "Heal" "HEAL" and "hEaL" the same.

And now we get into the wonderful world of scripting. I'm going to give the example in Lua, since it comes with MUSHclient, and I'm getting more familiar with that than the other languages.
^(/w) says, '(heal|rem/w+)'$ for the matching trigger.
Then in the scripting window, put this as send to script:
function autocleric( tname, tstr, wildcards )
  local spell = string.lower( wildcards[2] )
  if string.match( "removepoison", local ) == not nil then
    local = "remove poison"
  end -- find remove poison
  send( "cast '"..local.."' "..wildcards[1] )
end -- autocleric

I didn't test it out, as botting is illegal on my mud and I didn't want to accidentally get in trouble, but it should work.

Also, as a side note, please take a quick look at http://www.gammon.com.au/mushclient/funwithtriggers.htm for help with using regex wildcards.. The ones in your previous example "rem+ovepoison" would match on "remmmmmmmmmmovepoison" because the + matches one or more of the previous character. Regex is a little weird at first, but doesn't take long to get used to it.
Amended on Wed 28 Feb 2007 05:29 PM by Shaun Biggs
Australia Forum Administrator #8
I agree, although that example is slightly wrong. For one thing, things like /w should be \w, and for the player's name you need one or more so it should be \w+.

This trigger which uses inline (Lua) scripting, does what you are trying to do:


<triggers>
  <trigger
   enabled="y"
   ignore_case="y"
   match="^(\w+) says, '(heal|rem\w*)'\.$"
   regexp="y"
   send_to="12"
   sequence="100"
  >
  <send>
local spell = string.lower ("%2")  -- make spell lower case
if string.sub (spell, 1, 3) == "rem" then  -- rem something?
  local removepoison = string.sub ("removepoison", 1, string.len (spell))
  if spell ~= removepoison then
    return
  end -- they didn't say "removepoison"
  spell = "'remove poison'" -- change spell
end -- if

-- now cast it

Send ("cast " .. spell .. " %1")
</send>
  </trigger>
</triggers>



The trigger matches on "heal" or "rem<something>".

Then in the script I check that the <something> is the rest of the word "removepoison", truncated to the length of what they typed. It also has "ignore case" set to allow for the upper-case letter.

If they typed "removepoison" (or part of it), then it changes the spell to 'remove poison' (including the quotes), ready for casting.
USA #9
I always get / and \ confused... stupid lysdexia. And I think I found my mistakes on the other parts, but I fail to see how it differs from yours. Well, once mine is written without a stupid amount of errors. Here it is converted to be not a function.
Trigger:  ^(\w+) says, '(heal|rem\w+)'/.$

local spell = string.lower( "%2" )
if string.match( "removepoison", spell ) == not nil then
  spell = "'remove poison'"
end -- find remove poison
send( "cast "..spell.." %1".. )

This will set the spell to "remove poison" if the whole spell name matches within "removepoison" starting from the beginning, otherwise it doesn't change the spell string at all, allowing for a greater variety in the future, and does it without as much mucking about with creating new variables. The trigger itself takes care of the fact that it can potentially match on "rem" if it's for the poison. I also have a habit of encapsulating everything into functions.

As for the weird issues with variable names and confusing / and \... sorry about that. I should know better than to code while questing.
Amended on Thu 01 Mar 2007 12:11 AM by Shaun Biggs
Australia Forum Administrator #10
Yes, your method looks a bit neater once you get rid of the syntax and logic problems. :)

This is better:


if string.match( "removepoison", "^" .. spell ) then
  spell = "'remove poison'"
end -- find remove poison
send( "cast " .. spell .. " %1")


This line doesn't work as intended:


if string.match( "removepoison", spell ) == not nil then


The expression "not nil" is like saying "not false" and thus is true. Thus you are saying:


if string.match( "removepoison", spell ) == true then


The function string.match returns a string or nil, thus it will never return true.

You could have written:


if string.match( "removepoison", spell ) ~= nil then


But I think that looks redundant. Any non-nil (or non-false) result is considered true, so my version is cleaner, IMHO.

Finally, you don't want the 2 trailing dots on the "send" line.

Quote:

... if the whole spell name matches within "removepoison" starting from the beginning ...


I made sure it matched from the beginning by adding the "^" to the start of the regular expression.
USA #11
Quote:
if string.match( "removepoison", spell ) ~= nil then


But I think that looks redundant. Any non-nil (or non-false) result is considered true, so my version is cleaner, IMHO.
I think there's a big conceptual difference between testing for non-nil and testing for true. Nil means "no value", and that can be different from "false". You could imagine a ternary logic system of sorts where variables are true, false or unspecified.

Or, consider function variables. Imagine I have a function like so:

function f(flag)
  if flag then
    -- do something
  else
    -- do something else
  end
end


Let's say you want flag to default to true. So, if f is called with no arguments (like just f()) the parameter will be nil.

You could write:


function f(flag)
  flag = flag or true

  if flag then
    -- do something
  else
    -- do something else
  end
end


However, this will prevent the value of "flag" from being negative. So, you could do:


function f(flag)
  flag = (flag == nil and true) or flag
  if flag then
    -- do something
  else
    -- do something else
  end
end


This way, if flag is nil, then you take the value true; otherwise, if flag is non-nil (so flag==nil is false) you will take the value of flag.

This is, incidentally, why I also prefer to sometimes leave in the "true" in if statements:

if foo == true then ... end

It makes more sense, I think, when you are looking at negation:

if foo == false then ... end

Here, your test will fail when foo is nil. Perhaps this is not what you want, and arguably you should be testing for nil to make sure you have a value in the first place.

All this to say that I think there's quite a conceptual difference between nil (i.e. unspecified) and false (i.e., well, specified to be false).
Amended on Thu 01 Mar 2007 04:24 AM by David Haley
USA #12
Quote:
The expression "not nil" is like saying "not false" and thus is true.

I thought so too, until I couldn't figure out why one script wasn't working. After pounding my head on my monitor a few times (not normally a good option), I decided to try running this:
/print( false == false )
/print( false == nil )

the output for the first one is true. The output for the second one is false. I'd repeat what I said then, but I assume you frown upon long strings of expletives on your board. Quite counterintuitive if you're used to C, but then again Lua doesn't start arrays with 0 either (much to my constant confusion) string.match will return a nil value if a pattern is not found within the supplied string.

Your way makes a LOT more sense to read, but mine still does work. Once again, minus the syntax and logic problems. And I keep forgetting about the ~= and trying != instead. Crazy Lua. I keep thinking ~= looks like "sort of equals"
Amended on Thu 01 Mar 2007 05:09 AM by Shaun Biggs
Australia Forum Administrator #13
Quote:

This is, incidentally, why I also prefer to sometimes leave in the "true" in if statements:

if foo == true then ... end


I see your point, but I think such code ends up becoming a bit obscure.

The construct:


if x then
 --- some code
end 


... is going to execute "some code" if x is true.

My view is that saying:



if x == true then
 --- some code
end 


... is adding another level of "is true" tests when one is already implied. You may as well say (to take the idea to extremes):


if (x == true) == true then
 --- some code
end 


To give an example from natural language, I may ask "is it raining?". To say "is it true it is raining?" is a bit redundant (except for emphasis perhaps). And no-one says "is it true it is true that it is is raining?".

The Lua authors recommend the idiom:


a = b or c


... as a simple way of supplying a default. It falls down a little bit with booleans, because, as you say, it is hard to supply a negative.

In the Lua manual, they repeatedly recommend idioms like this:


if string.find(s, "^%d") then ...


... rather than:


if string.find(s, "^%d") ~= nil then ...


I think this is because string.find returns nil if there is no match, and they define an "if" as failing if it is passed nil as the condition.
Australia Forum Administrator #14
Quote:

/print( false == nil )

...

Quite counterintuitive if you're used to C


Yes, in C, under Windows, false and nil are probably going to end up being defined as the number 0. Thus, under C, under Windows, the following 3 are probably all equivalent:

  • 0 (zero)
  • FALSE
  • NIL


I qualify that a bit by adding that it probably depends a bit on the defines you are using. Also, on some platforms, NIL is not in fact zero.

However in Lua those are 3 distinct values, and types as well. One is a number, one is a boolean, and one is the special "non-value" nil. None are equivalent to each other.

You also have to be careful in Lua, that 0 is considered true, as the manual warns. Try this, for example:


if 0 then
 print "hi there"
end


C programmers do not expect to see anything printed.
Australia Forum Administrator #15
Quote:

Lua doesn't start arrays with 0 either ...


That is true, and is another catch for C programmers. Personally I quite like it, as you can use 1 for the first position in a string, and -1 for the last position, which makes sense to me.

I also think counting from 1 is more natural in real life, and bearing in mind Lua was designed as a scripting language, and not a mathematical language.

People do stuff like:

  1. Say "He is number 1 in his class!" (not number 0)
  2. Say "I will have the first turn" (not the zero'th turn)
  3. They number lists (like this), from 1.


I acknowledge there are arguments to be made for numbering from zero, so please don't do dozens of posts proving me wrong. There is something to be said for both sides of the argument, and this argument has been pursued on the Lua wiki as well, at some length. :)

For more information, see this:

http://lua-users.org/wiki/CountingFromOne

My thoughts appear there as well. ;-)
USA #16
Well, the only real argument is basically one that programatically the engine has to subtract one *then* start indexing things in memory. Its like what.. a few clock cycles? Maybe if we still coded for 1mhz 8086 processors it would be hugely critical. lol
USA #17
Quote:
That is true, and is another catch for C programmers. Personally I quite like it, as you can use 1 for the first position in a string, and -1 for the last position, which makes sense to me.

I think it makes sense to most people as well, I'm just so used to counting from 0 on a computer that I get snagged every now and again. I personally don't care either way, so long as I can remember which I'm using. Otherwise, I just don't notice the error for a while until I realize what language I'm using.

Quote:
I acknowledge there are arguments to be made for numbering from zero, so please don't do dozens of posts proving me wrong. There is something to be said for both sides of the argument, and this argument has been pursued on the Lua wiki as well, at some length.

This was not my intention, I was just using that as an example of how Lua is different from languages with which I am more familiar.

Quote:
You also have to be careful in Lua, that 0 is considered true, as the manual warns.

Also fell into that trap a while ago while converting a script for MC from jscript into Lua... I had only been using the language for a day or so. Again solved with me doing a painfully simple test to find out I fell victim to thinking things worked a certain way before looking them up. Fortunately, I've been poking a friend who has been helping me out when I get stuck, and I've had to respond to most of the solutions with "Oh, that was simple." I really do like the language. They combined simple and powerful quite well.
Amended on Thu 01 Mar 2007 07:18 PM by Nick Gammon
USA #18
Quote:
To say "is it true it is raining?" is a bit redundant (except for emphasis perhaps).
Well, that's my point. :-) I like to include it when I want to emphasize something. Of course I very often omit it, but occasionally when the context might not be perfectly clear I include it. And as I said, I think it makes more sense when you are speaking of the "false" case.

If I ask you, "is it not raining", I expect to hear "yes" if and only if it is not raining. If you don't know, that is, if your "value" for the proposition "raining" is "nil", I do not expect you to say yes. Rather I would expect something like "I don't know".

Of course in programming it is very convenient to have nil be the same as false for conditions. Nonetheless I think it's important to always, always remember that they are not the same thing. For instance, t = {}; t.foo = "bar"; t.foo = nil is quite different from assigning false to t.foo.

I think though that what we're saying is not fundamentally that different. I may have exaggerated my position somewhat; I certainly don't always throw in the == true. I only use it when for whatever reason it makes things clearer to me.

Quote:
Well, the only real argument is basically one that programatically the engine has to subtract one *then* start indexing things in memory. Its like what.. a few clock cycles? Maybe if we still coded for 1mhz 8086 processors it would be hugely critical. lol
I'm not sure this is an issue. It depends on how tables and arrays are implemented, I suppose.
USA #19
Your kidding me right? It has nothing to do with how tables or arrays are "implimented". Its just basic machine logic. If you have a block of say integer values:

300: AB CD 1D 34 5F DD 00 00

The code to get them is "on the machine level" going to look like:

ldx #0000
1:lda 0300, x
jsr <some place you do something with the number>
cmp #0000
inx
bne 1:
rts

This is true even if you are retrieving text strings. Unless you intentionally leave the first index location of your data structure empty, at some point your engine or code has to subtract 1 from the value, to get back the "correct" memory location to start retrieving the data. Otherwise you have areas in memory that are wasted, because processors "all" start with 0 as the first offset location for retrieving data from memory.

Dang modern programmers.. Don't know a damn thing about how the machine actually works... ;) lol
Australia Forum Administrator #20
Quote:

Nonetheless I think it's important to always, always remember that they are not the same thing.


I agree with this, the principle being the same as SQL where NULL represents "no data", even in a case of a boolean field, where the other values can be true or false.

Conceptually it is very useful to be able to say "nil means no data". Take for example, the question:

"Is David going to the party?"

Possible answers are:

  • Yes (true)
  • No (false)
  • Don't know (nil)


We could conceivably scan our table here and contact each person in our list for whom we have a "don't know" and attempt to convert it into "yes" or "no", so we can be sure how many people are attending.

For such an operation, distinguishing "don't know" (nil) from "no" (false) is obviously important.

On the other hand, to count how many people are attending the party (as far as we know), then a simple test for "anything that is not 'no' or 'nil'" gives us the answer. This is effectively what the Lua "if" test does.
Australia Forum Administrator #21
Quote:

Your kidding me right? It has nothing to do with how tables or arrays are "implimented". Its just basic machine logic.


No, I'm afraid he is not. Initially Lua implemented tables purely as hash-table lookups. In other words, the entry for [0] is a hash of 0, the entry for [1] is a hash of 1, and so on.

In fact in Lua there is absolutely no objection to having a negative index. Take for example:


t = {}

t [-5] = "Nick"
t [-10] = "Gammon"

table.foreach (t, print)

Output

-5 Nick
-10 Gammon


Now if they were using machine code to index into an array, as you suggested, then the negative entries would index out of the bounds of the table.

Thus, in early versions, the time taken to access any entry would be constant, that is the time needed to convert the subscript into a hash, and find it in the table. They could have started tables at 100, and not made any difference.

However in more modern versions of Lua (maybe 4 onwards), they have made table access more efficient by making tables internally have two parts - a hash part and a numeric part. Thus a table consisting of only subscripts 1 to 10 would fall entirely in the numeric part. If you added a subscript "x", or a large or negative number, then that goes into the hash part.

I think you would find that all this logic, like seeing what range the subscript is in, hashing it, and so on, would totally swamp any considerations (in CPU cycles) of whether you start at 0 or 1.
USA #22
Probably true, though, on the machine code level, its still starting with 0. You're just basically saying that the system they use in Lua has that 0 in some place that may be 500 addresses "before" the place currently being used in what ever buffer they created to hash the table. In which case, your 1 isn't even a one, its a numeric lookup to a memory address, in which case, it might as well be a, b, c, d, e, etc, for all that the actually value matters (mind you, numbers tending to be way more useful. lol)
Australia Forum Administrator #23
Quote:

... on the machine code level, its still starting with 0 ...


I think this is a matter of definition, a bit. Say I have a table like this:


t = {}
t ["nick"] = 10
t ["gammon"] = 20

table.foreach (t, print)


The keys are "nick" and "gammon". Which one does the table "start" with? It actually prints "gammon / nick" when I run it, so I suppose you can say it "starts" with gammon, but I think this is an idiosyncracy of the hashing algorithm.

The fact is, that in Lua (and various other constructs, for example some container types in STL), there is no linear array, and therefore no "first" entry in the classical sense (that is, the sense of "index by zero").

If you make a linked list in C, the first entry is the first item in the list, but in terms of memory storage, may be occupying a higher memory address than subsequent items.
Australia Forum Administrator #24
Quote:

Lua doesn't start arrays with 0 either ...


Following on from this, really Lua doesn't start arrays anywhere in particular, as I have shown, you can have negative subscripts.

It is probably more accurate to say that some Lua table access functions, for example table.foreachi, and the ipairs function, work by accessing item 1 onwards in a table, even if other values (like 0, -10, -2343) exist.

They are defined to do that, in other words, the function does that by definition, not because the table "starts at one".

Here is an excerpt from the Lua reference manual (see http://www.lua.org/manual/5.1/manual.html#5.1):


ipairs (t)

Returns three values: an iterator function, the table t, and 0, so that the construction

     for i,v in ipairs(t) do body end

will iterate over the pairs (1,t[1]), (2,t[2]), ···, up to the first integer key absent from the table.  


The ipairs function does not claim to start at the "start of" the table, but simple starts at index 1, and so on, until it finds a missing integer key.
USA #25
Quote:
Dang modern programmers.. Don't know a damn thing about how the machine actually works... ;) lol
Dang newbies, don't know a damn thing about how languages are actually implemented internally or any of the formal reasoning for why things work the way they do. *wry smile*

Nick has started explaining this and I will explain it some more. If your array is implemented as a hash table, then the key is for all intents and purposes not a number, even if it is "0" or "1" etc.

The implementation of a hash table will convert your key, using some magic, to a hash value. That hash value will then give you the bucket of the hash table. For simplicity's sake we will assume that there are no collisions in the hash table.

Now, there is no guarantee that the bucket assigned to key "1" by the hash table is the number "1" or "0" or any such thing. For all you know the bucket could be 792.

So, starting your array by zero or one makes absolutely no difference in terms of performance from the Lua programmer's side. You still have to go through the hash function in C, which will convert the key to a bucket. The bucket, of course, is a memory address, which can then be dereferenced immediately.

Therefore, I maintain my original statement and refute your response.

Your mistake is to have not understood the difference between a language's semantics, its implementation in some programming language e.g. C, and finally the machine code underlying it all. You did not realize that the implementation of a data structure can change, sometimes dramatically, the procedure to look up a given key. And finally I think you did not sufficiently mark the difference between a table key and an index into memory. The two are quite different, especially in a higher-level language like Lua the intention of which is to remove oneself from implementation details and establish a formal semantics independent of machine-level specifics.

And finally, I think you were a little premature in your supposition that I know nothing about how things actually work at the machine level. :-) Writing some stuff in hex and assembly doesn't necessarily make your point any better, especially when by focusing on the leaf, you missed the tree, much less the whole forest, as it were.



Besides, as Nick pointed out, even if you were translating numbers to indices -- as Lua does for the special part of the table that Nick mentioned -- the cost of subtracting one from a number is, for almost everybody and for almost all purposes, completely irrelevant when compared to other costs. I can think of rather few cases when you would actually care about squeezing every last machine instruction's worth of time out of your program, and in most of those cases, you would be somewhat silly to be doing it in Lua to begin with.



Anyhow. Enough of that (for now at least).
Quote:
On the other hand, to count how many people are attending the party (as far as we know), then a simple test for "anything that is not 'no' or 'nil'" gives us the answer. This is effectively what the Lua "if" test does.


That's a very good point Nick. It's true that in almost all cases, a value of "I don't know" is basically the same as "not yes". And usually, that is what we want. What it comes down to is the nature of the question asked.
USA #26
You don't need to explain it more. As I said in my own later post, in cases where you are using hash systems, the index is just a place holder. If its numeric you can, in theory, get a ordered list out of it. But it could just as easilly be colors, sizes, types of bugs, or some sequence of letters, like a-z, aa-az, ba-bz, etc. The "starting" symbol is completely irrelevant.

What I meant about the machine level stuff is "linear" tables, where the index is some specific number of records from the "start" of the memory its sitting in. Most languages tend to require, even with strings, which often have fixed limits on there allowed length, that the discrete "chunks" be a known size, so that the index number is "actually" a multiplier to the linear location of the *actual* memory address of the data. In other words, while ones like Lua uses hashes, because they are more efficient in some ways at storage, though maybe more costly in terms of some overhead, (a bit enough hash will be bigger than the processor can keep in its L1 or L2 caches for fast retrieval, for example), most languages have not used liked lists or hashes for arrays, but instead structured them like:

0000: [data chunk #1, fixed size, two bytes]
0002: [data chunk #2, fixed size, two bytes]
0004: [data chunk #3, fixed size, two bytes]
0006: [data chunk #4, fixed size, two bytes]

The location is literally n * 2 bytes into the arrays memory in such cases. The 3 position is either actually #4, or its (n-1) * 2, depending on if you use 1 or 0 to start the indexing.

See, all anyone had to do, which Nick did quite handilly, is explain that a hash was being used, not a linear address space. I freely admit I didn't know the implimentation differed from what "most" languages consider to be "arrays". So sue me. Your extra explanations was hardly necessary though, nor was the email I got from Shaun, which also covered the same thing. BTW, how are you supposed to correctly reply to those, since the dang things reply address ends up being the "forum" instead of the senders?

Oh, and I am well aware of what linked lists are too. I tried, unsuccessfully, one time to make one that could be both sorted and randomized at the same time. I strongly suspect that the useless Pascal compiler I was using was taking up too much memory on the machine and the 64k limit on the old Apple IIs was only compounding the problem. It *ran*, but large parts of the list got "lost" somehow... Wasn't my best success story. lol
USA #27
Quote:
Most languages tend to require, even with strings, which often have fixed limits on there allowed length, that the discrete "chunks" be a known size, so that the index number is "actually" a multiplier to the linear location of the *actual* memory address of the data.
You are compounding high-level language semantics and implementation details. C or assembly working a certain way means next to nothing about how other languages present the data to you and what they require chunks or what-have-you to look like.

I don't understand how can you make claims about what "most languages [...] require" when you only know a few small number to begin with (and not the most representative languages, to boot) much less implementation details of e.g. interpreted languages. It's like your claim about "most languages" considering it an error to have expressions as statements. You really shouldn't go out on limbs like that...

Quote:
I freely admit I didn't know the implimentation differed from what "most" languages consider to be "arrays". So sue me.
Well, I'm not sure what basis you have to be making claims about "most" languages to begin with... so yes, I will in fact sue you. :-) You have a tendency to make claims about things outside the scope of what you truly know, if you don't mind me saying so.

Quote:
In other words, while ones like Lua uses hashes, because they are more efficient in some ways at storage,
Hashes are not used for storage efficiency. In fact they are not necessarily the most storage-efficient data structure! Hashes are used for the extremely fast look-up time. (Of course, assuming that your hashing algorithm is good, and that you have enough buckets and/or few enough elements to avoid collision.)

Quote:
BTW, how are you supposed to correctly reply to those, since the dang things reply address ends up being the "forum" instead of the senders?
I thought those emails had as a return value the sender's email address; at least, the ones I received did. In any case you can probably just use the forum's email facility to send an email back, if you care to do so.
USA #28
Quote:
Your extra explanations was hardly necessary though, nor was the email I got from Shaun, which also covered the same thing. BTW, how are you supposed to correctly reply to those, since the dang things reply address ends up being the "forum" instead of the senders?

There is a spot if you click on someone's name which says "Send an email to _____" which is what I used to email you in the first place. Sorry about dealing with things like that, but I was trying to avoid this whole thread, as I said in the email. Nick said he didn't want the whole 0 vs. 1 thing cluttering up his boards when there was already a link to a very long discussion on Lua's site.

Quote:
Oh, and I am well aware of what linked lists are too. I tried, unsuccessfully, one time to make one that could be both sorted and randomized at the same time. I strongly suspect that the useless Pascal compiler I was using was taking up too much memory on the machine and the 64k limit on the old Apple IIs was only compounding the problem. It *ran*, but large parts of the list got "lost" somehow... Wasn't my best success story. lol

I did the same in Pascal during high school on my 286.. how I miss that 12mhz and a whole meg of ram and HUGE 40 mb hard drive. Anyway, I wound up having to make a doubled list. I had the data, the next and previous. Then a record that contained the id and the next and previous for that set. It worked really well until I attempted to delete something, where it would reboot my computer. I never could figure out where I was going wrong.

As for the 64k limit, I believe that was a Pascal issue. Pascal was designed as a teaching language, and has a few oddities because of that. If I remember right, there was a limit in how large a source file could be. You simply have to break your code up into smaller files. I forget what that the syntax for that is, but when you want to access the extra file, you put it in your Uses section. Doing that, I made a game based around the doors game "the Pit" for my CS final my senior year which took up around 150k of source, and then several data files. Had I actually known known some of the things I know, it would probably have been about half that size.
Amended on Fri 02 Mar 2007 07:44 PM by Nick Gammon
USA #29
Actually, I think I got it wrong. It wasn't 64k, but... Well, it had to do with the limitations of the hardware. Each "bank" was only 0000-FFFF, or "640k", but you generally only really had access to one bank of the two, and a number of memory sections where dedicated to video display, the stack, the softswitches used to control the hardware and the permanent ROM functions. That left you with 3-4 usable "sections", non-connected with each other, none of them bigger than about maybe 300k? And some of that space had to be taken up by the debugger and reload code for the Pascal system when you exited your application.

As for hashes. Sorry, partly meant lookup speed, but also the fact that, while sections of memory might go unused, the chunks don't "necessarilly", I assume, need to be fixed sizes, so you use "less" space for data that doesn't need fixed sizes in it. Unless I am completely wrong about that. Needing to pre-allocate something like 200 strings, all of them 255 bytes in length, because you can't just store the lookup address (and maybe length), then use only the space needed to store *that* specific string.... Gets rather inefficient in terms of memory use. Though, I admit I don't know if hashes help to solve that problem or not, but it just seems reasonable that, to save on space, you could/would do something like that.
USA #30
Quote:
but also the fact that, while sections of memory might go unused, the chunks don't "necessarilly", I assume, need to be fixed sizes, so you use "less" space for data that doesn't need fixed sizes in it. Unless I am completely wrong about that.
Well... :-)

What you're really talking about here is the difference between pointers and storing the actual data. If I had an array of pointers, (hint: a list of buckets) then I don't suffer from data elements that are larger than others in the sense that I don't have to preallocate all necessary space.

But this has nothing to do with hash tables per se.

One advantage of hash tables over arrays regarding storage efficiency is that if you have a very sparse array you are wasting lots of space. E.g., if all you have is a[1], a[50], a[102] then it would be wasteful to store than in an array of size 102 because you'd have 99 empty slots. If you had a hash table with a reasonably sized (or dynamically growing) bucket list you would be using a lot less.

Quote:
Though, I admit I don't know if hashes help to solve that problem or not, but it just seems reasonable that, to save on space, you could/would do something like that.
Yes, it's a reasonable thing to do, which is what I was referring to just above here regarding pointers. Still it is not a property of hash tables and is rather the difference between storing pointers to data versus the actual data.
Australia Forum Administrator #31
Quote:

... all anyone had to do, which Nick did quite handilly, is explain that a hash was being used, not a linear address space ...


I don't really want to get bogged down here with claims and counter claims about what people know, or don't know.

The forum is intended to be an informational and educational tool, and thus I am happy to clarify things like that.

It is easy - if you have come from a history, as I have, of using simpler languages (if you can call assembler simple) - to assume that language implementors still do things the same way as they always have. Which in a sense they do, deep down.

You can see from the fact that what look like arrays in Lua, but support negative subscripts, must really use something else.

Interestingly, the Standard Template Library (STL), when storing similar things (in a map or set), does not actually use hashes but a B-Tree, which has the interesting property that regardless of the order in which you put things, they come out in sequence (alphabetic order for strings).

If you expected the keys to be hashed you would be surprised by that.

It shows that taking an interest in the implementation details can clarify some confusion, when the behaviour of the overall algorithm is not quite what you expect.
Amended on Fri 02 Mar 2007 07:53 PM by Nick Gammon
Australia Forum Administrator #32
Oh, and how is the Autocleric going?
USA #33
Quote:
Interestingly, the Standard Template Library (STL), when storing similar things (in a map or set), does not actually use hashes but a B-Tree, which has the interesting property that regardless of the order in which you put things, they come out in sequence (alphabetic order for strings).
This is indeed a pretty interesting design decision on their part, mainly because it requires that whatever you are storing be sortable/comparable. If you're inserting pointers, then everything is fine because you can compare pointer addresses. String objects also come with the comparison operator (lexicographical comparison). But if you're trying to key on a new data structure (or store it in a set) you need to have your own comparison.

Of course, you have the same problem with hash functions, in that it's unclear how you would go about hashing an unknown data structure without additional information about it.

So I'm guessing the btree choice had to do not only with wanting the properties of binary trees (or not), but also with how to actually implement a binary tree versus a hash table. As soon as you stop storing pointers you need to worry about your "keying" function, and it's probably easier to define an ordering on things than to define a hash function -- because, in part, a hash function has to be much more carefully crafted in order to distribute elements more or less equally over the buckets and not have (too many) collisions.
USA #34
Quote:
This is indeed a pretty interesting design decision on their part, mainly because it requires that whatever you are storing be sortable/comparable. If you're inserting pointers, then everything is fine because you can compare pointer addresses. String objects also come with the comparison operator (lexicographical comparison). But if you're trying to key on a new data structure (or store it in a set) you need to have your own comparison.

I'm pretty sure the Lua developers came up with on their own. Comparison by strings is definitely not what is going on, because you can have functions as keys and values for tables in Lua.
USA #35
They use a hash function, not an ordering. I'm guessing that since Lua has a relatively small number of pure data types, it's possible to code in advance the hash function for all those types. But right, a straight comparison is definitely not what they use.
Australia Forum Administrator #36
Interestingly, I think you will find that all Lua types can be readily hashed because they are stored internally as a union. See the source for this:


/*
** Union of all Lua values
*/
typedef union {
  GCObject *gc;
  void *p;
  lua_Number n;
  int b;
} Value;



Depending on the size of lua_Number, this value is going to be a small size (say around 4 bytes, maybe slightly more). Lua internally stores strings in such a way that identical strings are stored in the same place. Thus if the *address* of two strings is the same, the strings are the same. Similarly for tables, which are really a pointer to the table. And, userdata is simply an address. Once you take the type of data into account, in addition to the value, you have a small field that needs to be hashed to index into a table.
USA #37
Oh, that is interesting. I guess it's what makes the most sense, too. :-)

I'm guessing that lua_Number is closer to 8 bytes, since the default number size is a double.

I wonder if their hashing function simply operates on a sequence of bytes. After all, it appears that all of their values are 4 to 8 bytes (the pointer sizes depend on 32 vs 64 bit).

One of these days, I'd like to really look through the Lua implementation and see what it's like under the hood...