This forum is a read-only archive of the Gammon Software forum (2000–2026). No new posts can be made. Search the archive.

Epic Failure With Copyover (Linux)

Posted by YoshoFyre on Mon 02 Aug 2010 06:49 AM — 18 posts, 87,661 views.

YoshoFyre #0
so every time i attempt to hotreboot the mud, when it restarts i get the following error after trying to reconnect the clients.

Mon Aug 2 02:34:48 2010 :: MUD is ready to rock on ports 6666 and 8080.
Mon Aug 2 02:35:17 2010 :: Sock.sinaddr: 192.168.1.210
Mon Aug 2 02:35:23 2010 :: Loading Character.
Mon Aug 2 02:35:26 2010 :: Character@192.168.1.210 has connected.
Init socket: bind: Address already in use

this causes the mud to crash out...

any suggestions?
Nick Gammon Australia Forum Administrator #1
It looks like the copyover (hotboot) processing didn't release the connection (the mud "binds" to the port) so afterwards it found the port in use (probably by itself).

It's hard to help more without knowing what version of ROM you are using, where you got the copyover from, and so on.
YoshoFyre #2
it started as Rom2.4b ... but its been heavily modified...

im also unsure of the copyover file... i originally modded the mud back in 2001 under cgywin. i have recently reloaded the code from backup and now am running under linux. when the code was running under cygwin, the mud would hotreboot once... and crash the second time... i never really had a chance to debug it more past then.

here is the comment from the source file for copyover... if that helps at all


/*

You need to define:

COPYOVER_FILE - temporary data file used
EXE_FILE - file to be exec'ed (i.e. the MUD)


Note that I advance level 1 chars to level 2 - this is necessary in MERC and
Envy, but I think that ROM saves level 1 characters too.

Note that you might want to change your close_socket() a bit. I have changed
the connected state so that negative states represent logging-in, while as
positive ones represent states where the character is already inside the game.
close_socket() frees that chararacters with negative state, but just loses
link on those with a positive state. I believe that idea comes from Elwyn
originally.

Things to note: This corresponds to a reboot, followed by the characters
logging in again. This means that stuff like corpses, disarmed weapons etc.
are lost, unless you save those to the pfile. You should probably give the
players some warning before doing a copyover.

The command was inspired by the discussion on merc-l about how Fusion's MUD++
could reboot without players having to re-login :)

*/

i noticed that it suggests to change close_socket() but it doesn't go in to detail as to what i need to do...

thanks
Nick Gammon Australia Forum Administrator #3
The inability to bind refers to the single "listening" socket - the one that accepts connections (eg. to port 4000). The other sockets are the ones established per-player.

At this stage I would have to guess that the copyover does not close the listening socket, thus later when it tries to re-open it, it is still in use. It doesn't hurt to close this socket as you don't send player data down it, it simply gets new connection notifications.
YoshoFyre #4
Nick Gammon said:

The inability to bind refers to the single "listening" socket - the one that accepts connections (eg. to port 4000). The other sockets are the ones established per-player.

At this stage I would have to guess that the copyover does not close the listening socket, thus later when it tries to re-open it, it is still in use. It doesn't hurt to close this socket as you don't send player data down it, it simply gets new connection notifications.
so where would i put that... in do_copyover after the execl ? or in copyover_recover near the beginning???

I know socket_close(d) kills the player, but how do i kill the listening port?
Amended on Tue 03 Aug 2010 12:05 AM by YoshoFyre
Nick Gammon Australia Forum Administrator #5
It's the control port. In Smaug it looks like this:


   boot_db( fCopyOver );
   log_string( "Initializing socket" );
   if( !fCopyOver )  /* We have already the port if copyover'ed */
      control = init_socket( port );


Note how the control socket is not initialized on a copyover.
YoshoFyre #6
Nick Gammon said:

It's the control port. In Smaug it looks like this:


   boot_db( fCopyOver );
   log_string( "Initializing socket" );
   if( !fCopyOver )  /* We have already the port if copyover'ed */
      control = init_socket( port );


Note how the control socket is not initialized on a copyover.

Thanks a bunch!! that worked...

but now i have new problem... after the copyover, i can't get anymore incomming connections. everyone who was connected remains, but when i try to bring a new client in, i get "connected" but it just sits there and does nothing...
Nick Gammon Australia Forum Administrator #7
That happened to me a while back, and it is quite frustrating to fix. Certainly it will be hard without seeing the code or having access to debugging it.

Basically there is code in the comm.c along these lines:


   /*
    * Poll all active descriptors.
    */
   FD_ZERO( &in_set );
   FD_ZERO( &out_set );
   FD_ZERO( &exc_set );
   FD_SET( ctrl, &in_set );
   maxdesc = ctrl;
   newdesc = 0;

  // add in active descriptors to the sets here

   if( select( maxdesc + 1, &in_set, &out_set, &exc_set, &null_time ) < 0 )
   {
      perror( "accept_new: select: poll" );
      exit( 1 );
   }


Now if the "sets" are not set up right, or you pass the wrong number to "select" then it doesn't realize there is activity on that particular descriptor. I would check (perhaps in gdb) that these are being set up right.
Amended on Tue 03 Aug 2010 05:13 AM by Nick Gammon
YoshoFyre #8
here is mine from comm.c

/*
	 * Poll all active descriptors.
	 */
	FD_ZERO( &in_set  );
	FD_ZERO( &out_set );
	FD_ZERO( &exc_set );
	FD_SET( control, &in_set );
	maxdesc	= control;
	for ( d = descriptor_list; d; d = d->next )
	{
	    maxdesc = UMAX( maxdesc, d->descriptor );
	    FD_SET( d->descriptor, &in_set  );
	    FD_SET( d->descriptor, &out_set );
	    FD_SET( d->descriptor, &exc_set );
	}

	if ( select( maxdesc+1, &in_set, &out_set, &exc_set, &null_time ) < 0 )
	{
	    perror( "Game_loop: select: poll" );
	    exit( 1 );
	}

	/*
	 * New connection?
	 */
	if ( FD_ISSET( control, &in_set ) )
	    init_descriptor( control );
Nick Gammon Australia Forum Administrator #9
All I can suggest at this point is to fire up gdb and try to work out what is happening (particularly to the control port) during a copyover.

http://www.gammon.com.au/gdb
YoshoFyre #10
ok i have gdb installed, and i see how to use it when a program crashes out but i don't see how to get info from it when the program is still running and the program errors but doesn't crash...

how would i examine the control port?

If it helps any... here is pastebin with my copyover function and copyover recover function

http://pastebin.com/raw.php?i=5zWu5Pu9
Amended on Wed 04 Aug 2010 12:25 AM by YoshoFyre
Nick Gammon Australia Forum Administrator #11
Speedline02VA said:

ok i have gdb installed, and i see how to use it when a program crashes out but i don't see how to get info from it when the program is still running and the program errors but doesn't crash...


You start up under gdb. Something like this:


cd ../area
gdb ../src/rom

# set breakpoints now

run 4000    # start up - argument (port) is 4000

YoshoFyre #12
Nick Gammon said:

Speedline02VA said:

ok i have gdb installed, and i see how to use it when a program crashes out but i don't see how to get info from it when the program is still running and the program errors but doesn't crash...


You start up under gdb. Something like this:


cd ../area
gdb ../src/rom

# set breakpoints now

run 4000    # start up - argument (port) is 4000



I gotcha. i am that far, what i can't figure out is where to set the break points... i guess i should set points for all lines between the socket connection to the welcome screen... since i hang up somewhere between there?

upon further review, if i do a copyover, and then attempt to type anything in my ssh window while the mud is running, it crashes out with

New_descriptor: accept: Socket operation on non-socket
web-accept: Socket operation on non-socket
Amended on Wed 04 Aug 2010 02:23 AM by YoshoFyre
Nick Gammon Australia Forum Administrator #13
Back to basics, first. Is this Cygwin, or Linux? Or something else?

I wouldn't set a breakpoint "for all lines". Breakpoints stop execution at some point you consider important. After the breakpoint is hit (if it is hit) then you can step forwards, or examine variables, a line at a time, to confirm some hypothesis you might have.

Like, on the "select" line above, you might put a breakpoint and print what the value of maxdesc is.
YoshoFyre #14
Nick Gammon said:

Back to basics, first. Is this Cygwin, or Linux? Or something else?

I wouldn't set a breakpoint "for all lines". Breakpoints stop execution at some point you consider important. After the breakpoint is hit (if it is hit) then you can step forwards, or examine variables, a line at a time, to confirm some hypothesis you might have.

Like, on the "select" line above, you might put a breakpoint and print what the value of maxdesc is.
I'm running Ubuntu linux

Linux mud 2.6.22-14-server #1 SMP Sun Oct 14 23:34:23 GMT 2007 i686

Distributor ID: Ubuntu
Description:    Ubuntu 7.10
Release:        7.10
Codename:       gutsy

GCC version 3.4



so here is the connection part of my comm.c

/* Are we recovering from a Hot Reboot? */
if (argv[3] && argv[3][0])
{
  fCopyOver = TRUE;
  //control = atoi(argv[4]);
  if( !fCopyOver )  /* We have already the port if copyover'ed */
  {
    control = init_socket( argv[4] );
    wwwcontrol = init_socket( wwwport );
  }
}
else
  fCopyOver = FALSE;
if (!fCopyOver)
{
  control = init_socket( port );
  wwwcontrol = init_socket( wwwport );
}
boot_db();
if (!fCopyOver)
  init_web(7001);
sprintf( log_buf, "MUD is ready to rock on ports %d and %d.", port, wwwport );
log_string( log_buf );   
if (fCopyOver)
  copyover_recover();
game_loop_unix( control, wwwcontrol );
shutdown_web();
close (wwwcontrol);
//if(!fCopyOver)
  //close (control);


all the commented out stuff i did many years ago...

the code in bold i just modded per earlier

Edit: it appears that when i searched for the lines of code i modded with if (!fcopyover) i did not notice that i have that... so something must be wrong with control = init_socket( argv[4] ); to make the mud crash under copyover...

perhaps i should edit it back, and then set the break point to that line? thoughts?
Amended on Thu 05 Aug 2010 12:23 AM by YoshoFyre
Nick Gammon Australia Forum Administrator #15
What is the point of this?


 fCopyOver = TRUE;
  //control = atoi(argv[4]);
  if( !fCopyOver )  /* We have already the port if copyover'ed */
  {
    control = init_socket( argv[4] );
    wwwcontrol = init_socket( wwwport );
  }


You set fCopyOver to TRUE and then immediately test if it is true or not.
YoshoFyre #16
Nick Gammon said:

What is the point of this?


 fCopyOver = TRUE;
  //control = atoi(argv[4]);
  if( !fCopyOver )  /* We have already the port if copyover'ed */
  {
    control = init_socket( argv[4] );
    wwwcontrol = init_socket( wwwport );
  }


You set fCopyOver to TRUE and then immediately test if it is true or not.
i wasn't paying attention when i added if (!copyover) from a few posts back...

so the code is back to

  fCopyOver = TRUE;
  //control = atoi(argv[4]);
  control = init_socket( argv[4] );
  wwwcontrol = init_socket( wwwport );

and i crash on recover with the bind issue. when i get time later on today, i'll gdb the control line and see what happens
YoshoFyre #17
Problem Solved!!! Thanks for all the help... i'm sure i'll be back for a few more questions along the way!

heres my final code

  /* Are we recovering from a Hot Reboot? */
  if (argv[3] && argv[3][0])
  {
    fCopyOver = TRUE;
    control = atoi(argv[4]);
    //wwwcontrol = init_socket( wwwport ); //lets comment this out for a bug test
  }
  else
    fCopyOver = FALSE;

/*********************************************************/

  if (!fCopyOver) //--- we don't need to set the controls if its hot reboot
  {
    control = init_socket( port );
    wwwcontrol = init_socket( wwwport );
  }
  boot_db();