cafe-grader-judge-scripts Commit

cafe-grader-judge-scripts

Commit `r254:8d92eda43960` public

parent | child

Description:

- new install script - add comment in grader chain

Commit status:

[Not Reviewed]

References:

default

Diff options:

| | | |

Comments:

0 Commit comments 0 Inline Comments

Unresolved TODOs:

There are no unresolved TODOs

Add another comment

r254:8d92eda43960 - Wed, 11 Jul 2018 10:33:10 - 4 files changed: 48 inserted, 41 deleted

installer/install.sh ¶ +16 -20

               #!/bin/sh
+            + #installation script for cafe-grader, for ubuntu 16.04
+            +
               echo "This script will install and configure Cafe grader."
-            - RUBY_VERSION=2.1.2
+            + RUBY_VERSION=2.3.7
-            - echo "This will install Ruby $RUBY_VERSION under RVM"
-            - echo "Installing required apts"
-            - sudo apt-get update
-            - sudo apt-get install mysql-server mysql-client \
-            -   g++ gcc apache2 libmysqlclient20 build-essential \
-            -   git-core openssl libreadline6 libreadline6-dev \
-            -   zlib1g zlib1g-dev libssl-dev libyaml-dev libsqlite3-dev \
-            -   sqlite3 libxml2-dev libxslt-dev autoconf libc6-dev \
-            -   ncurses-dev automake libtool bison subversion \
-            -   pkg-config curl nodejs unzip pyflakes ruby default-jdk \
-            -   libmysqld-dev mercurial python-setuptools python-dev python3-numpy
-            - echo "Installing RVM"
-            - curl -k -L https://get.rvm.io | bash -s stable
-            - source ~/.rvm/scripts/rvm
               echo "Installing Ruby $RUBY_VERSION in RVM"
               rvm install $RUBY_VERSION
               rvm use $RUBY_VERSION
               echo "Fetching Cafe Grader from Git repositories"
               echo "Fetching web interface"
               mkdir cafe_grader
               cd cafe_grader
-            - git clone -q git://github.com/jittat/cafe-grader-web.git web
+            + git clone -q git://github.com/cafe-grader-team/cafe-grader-web.git web
               echo "Configuring rails app"
               cp web/config/application.rb.SAMPLE web/config/application.rb
               cp web/config/initializers/cafe_grader_config.rb.SAMPLE web/config/initializers/cafe_grader_config.rb
               #replace UTC in application.rb with the system timezone
               timezone='UTC'
               if [ -f '/etc/timezone' ]; then
                 timezone=\"`cat /etc/timezone`\"
               else
                 if [ -f '/etc/sysconfig/clock' ]; then
                   timezone=`grep -e '^TIMEZONE' /etc/sysconfig/clock | grep -o -e '\".*\"'`
                 fi
               fi
               replace="s!'UTC'!$timezone!g"
               sed -i $replace web/config/application.rb
               echo "At this point we will need MySQL user and database."
               echo "Have you created MySQL user and database for Cafe grader? (Y/N) "
               read ch
               if [ "$ch" = "n" -o "$ch" = "N" ]
               then
                 echo "Please open another terminal and create the user and database for Cafe grader."
                 echo "Don't forget to grant access to that database for the user."
                 echo "Please have username, password, and database name ready before continue."
                 echo
                 echo "The following are instructions:"
                 echo "1. Run mysql:"
                 echo
                 echo "      mysql -u root -p"
                 echo
                 echo "   if you have just installed mysql, the root password is the one that you have just entered"
                 echo "2. Create a new database, a new user, and grant access to grader database:"
                 echo
                 echo "      create user 'USERNAME'@'localhost' identified by 'PASSWORD';"
                 echo "      create database \`DATABASENEME\`;"
                 echo "      grant all on \`DATABASENAME\`.* to 'USERNAME'@'localhost';"
                 echo
                 echo "   Replace USERNAME, PASSWORD, and DATABASENAME accordingly."
                 echo
                 echo "Hit enter when ready..."
                 read dummy
               fi
               CAFE_PATH=`pwd`
               read database
               echo "Please provide grader username:"
               read username
               echo "Please provide $username password:"
               read password
               echo "development:" > config/database.yml
               echo "  adapter: mysql2" >> config/database.yml
               echo "  encoding: utf8" >> config/database.yml
               echo "  reconnect: false" >> config/database.yml
               echo "  database: $database" >> config/database.yml
               echo "  pool: 5" >> config/database.yml
               echo "  username: $username" >> config/database.yml
               echo "  password: $password" >> config/database.yml
               echo "  host: localhost" >> config/database.yml
               echo "  socket: /var/run/mysqld/mysqld.sock" >> config/database.yml
               echo "" >> config/database.yml
               echo "production:" >> config/database.yml
               echo "  adapter: mysql2" >> config/database.yml
               echo "  encoding: utf8" >> config/database.yml
               echo "  reconnect: false" >> config/database.yml
               echo "  database: $database" >> config/database.yml
               echo "  pool: 5" >> config/database.yml
               echo "  username: $username" >> config/database.yml
               echo "  password: $password" >> config/database.yml
               echo "  host: localhost" >> config/database.yml
               echo "  socket: /var/run/mysqld/mysqld.sock" >> config/database.yml
               echo "Object.instance_eval{remove_const :GRADER_ROOT_DIR}" >> config/initializers/cafe_grader_config.rb
               echo "Object.instance_eval{remove_const :GRADING_RESULT_DIR}" >> config/initializers/cafe_grader_config.rb
               echo "GRADER_ROOT_DIR = '$CAFE_PATH/judge'" >> config/initializers/cafe_grader_config.rb
               echo "GRADING_RESULT_DIR = '$CAFE_PATH/judge/result'" >> config/initializers/cafe_grader_config.rb
               echo "Installing required gems"
               gem install bundler
               bundle install
               echo "Running rake tasks to initialize database"
               rake db:migrate
               rake db:seed
               echo "Running rake tasks to precompile the assets"
               rake assets:precompile
+            + echo "setup the secret file"
+            + SECRET_A=`rake secret`
+            + SECRET_B=`rake secret`
+            + SECRET_C=`rake secret`
+            + echo "development:" > config/secrets.yml
+            + echo "  secret_key_base: '$SECRET_A'" >> config/secrets.yml
+            + echo "test:" >> config/secrets.yml
+            + echo "  secret_key_base: '$SECRET_B'" >> config/secrets.yml
+            + echo "production:" >> config/secrets.yml
+            + echo "  secret_key_base: '$SECRET_C'" >> config/secrets.yml
+            +
               echo "Intalling web interface complete..."
               echo
               echo "Fetching grader"
               cd ..
               mkdir judge
               cd judge
-            - git clone -q git://github.com/jittat/cafe-grader-judge-scripts.git scripts
+            + git clone -q git://github.com/cafe-grader-team/cafe-grader-judge-scripts.git scripts
               mkdir raw
               mkdir ev-exam
               mkdir ev
               mkdir result
               mkdir log
               echo "Configuring grader"
               cp scripts/config/env_exam.rb.SAMPLE scripts/config/env_exam.rb
               cp scripts/config/env_grading.rb.SAMPLE scripts/config/env_grading.rb
               # create new environment.rb file
               echo "RAILS_ROOT = '$CAFE_PATH/web'" > scripts/config/environment.rb
               echo "GRADER_ROOT = '$CAFE_PATH/judge/scripts'" >> scripts/config/environment.rb
               echo "require File.join(File.dirname(__FILE__),'../lib/boot')" >> scripts/config/environment.rb
               echo "require File.dirname(__FILE__) + \"/env_#{GRADER_ENV}.rb\"" >> scripts/config/environment.rb
               # compiling box
               MACHINE_TYPE=`uname -m`
               if [ ${MACHINE_TYPE} == 'x86_64' ]; then
                 gcc -std=c99 -o scripts/std-script/box scripts/std-script/box64-new.c
               else
                 g++ -o scripts/std-script/box scripts/std-script/box.cc
               fi
               cd ..
               echo "Now you are ready to run cafe grader...."
               echo
               echo "Try:"
               echo
               echo "  cd web"
               echo "  rails s"
               echo
               echo "and access web at http://localhost:3000/"
               echo "The root username is 'root', its password is 'ioionrails'."

lib/engine.rb ¶ +17 -9

               require 'fileutils'
               require File.join(File.dirname(__FILE__),'dir_init')
               module Grader
                 #
                 # A grader engine grades a submission, against anything: a test
                 # data, or a user submitted test data.  It uses two helpers objects:
                 # room_maker and reporter.
                 #
                 class Engine
+            +
                   attr_writer :room_maker
                   attr_writer :reporter
                   def initialize(options={})
                     # default options
                     if not options.include? :room_maker
                       options[:room_maker] = Grader::SubmissionRoomMaker.new
                     end
                     if not options.include? :reporter
                       options[:reporter] =  Grader::SubmissionReporter.new
                     end
                     @config = Grader::Configuration.get_instance
                     @room_maker = options[:room_maker]
                     @reporter = options[:reporter]
                   end
+            +
                   # takes a submission, asks room_maker to produce grading directories,
                   # calls grader scripts, and asks reporter to save the result
                   def grade(submission)
                     current_dir = FileUtils.pwd
                     user = submission.user
                     problem = submission.problem
                     begin
                       # TODO: will have to create real exception for this
                       if user==nil or problem == nil
                         @reporter.report_error(submission,"Grading error: problem with submission")
                         raise "engine: user or problem is nil"
                       end
                       # TODO: this is another hack so that output only task can be judged
                       if submission.language!=nil
                         language = submission.language.name
                         lang_ext = submission.language.ext
                       else
                         language = 'c'
                         lang_ext = 'c'
                       end
                       # This is needed because older version of std-scripts/compile
                       # only look for c++.
                       if language == 'cpp'
                         language = 'c++'
                       end
                       # COMMENT: should it be only source.ext?
                       if problem!=nil
                         source_name = "#{problem.name}.#{lang_ext}"
                       else
                         source_name = "source.#{lang_ext}"
                       end
                       grading_dir = @room_maker.produce_grading_room(submission)
                       @room_maker.save_source(submission,source_name)
                       problem_home = @room_maker.find_problem_home(submission)
                       # puts "GRADING DIR: #{grading_dir}"
                       # puts "PROBLEM DIR: #{problem_home}"
                       if !FileTest.exist?(problem_home)
                         puts "PROBLEM DIR: #{problem_home}"
                         raise "engine: No test data."
                       end
+            +         # copy the source script, using lock
                       dinit = DirInit::Manager.new(problem_home)
+            +         # lock the directory and copy the scripts
                       dinit.setup do
                         copy_log = copy_script(problem_home)
                         save_copy_log(problem_home,copy_log)
                       end
+            +
                       call_judge(problem_home,language,grading_dir,source_name)
                       @reporter.report(submission,"#{grading_dir}/test-result")
+            +         # unlock the directory
                       dinit.teardown do
                         copy_log = load_copy_log(problem_home)
                         clear_copy_log(problem_home)
                         clear_script(copy_log,problem_home)
                       end
                     rescue RuntimeError => msg
                       @reporter.report_error(submission, msg)
                       puts "ERROR: #{msg}"
                     ensure
                       @room_maker.clean_up(submission)
                       Dir.chdir(current_dir)   # this is really important
                     end
                   end
+            +
                   protected
+            +
                   def talk(str)
                     if @config.talkative
                       puts str
                     end
                   end
+            +     #change directory to problem_home
+            +     #call the "judge" script
                   def call_judge(problem_home,language,grading_dir,fname)
                     ENV['PROBLEM_HOME'] = problem_home
                     ENV['RUBYOPT'] = ''
+            +
                     talk grading_dir
                     Dir.chdir grading_dir
                     script_name = "#{problem_home}/script/judge"
                     cmd = "#{script_name} #{language} #{fname}"
                     talk "CMD: #{cmd}"
                     warn "ERROR: file does not exists #{script_name}" unless File.exists? script_name
                     system(cmd)
                   end
                   def get_std_script_dir
                     GRADER_ROOT + '/std-script'
                   end
+            +     #copy any script presented in std-script directory that is not in the problem_home
+            +     #this allow a problem setter to provide their own version for each script
+            +     #in case that they want to hack something
                   def copy_script(problem_home)
                     script_dir = "#{problem_home}/script"
                     std_script_dir = get_std_script_dir
                     raise "engine: std-script directory not found" if !FileTest.exist?(std_script_dir)
                     scripts = Dir[std_script_dir + '/*']
+            +
                     copied = []
                     scripts.each do |s|
                       fname = File.basename(s)
                       next if FileTest.directory?(s)
                       if !FileTest.exist?("#{script_dir}/#{fname}")
                         copied << fname
                         FileUtils.cp(s, "#{script_dir}", :preserve => true)
                       end
                     end
+            +
                     return copied
                   end
                   def copy_log_filename(problem_home)
                     return File.join(problem_home, '.scripts_copied')
                   end
                   def save_copy_log(problem_home, log)
                     f = File.new(copy_log_filename(problem_home),"w")
                     log.each do |fname|
                       f.write("#{fname}\n")
                     end
                     f.close
                   end
+            +
                   def load_copy_log(problem_home)
                     f = File.new(copy_log_filename(problem_home),"r")
                     log = []
                     f.readlines.each do |line|
                       log << line.strip
                     end
                     f.close
                     log
                   end
                   def clear_copy_log(problem_home)
                     File.delete(copy_log_filename(problem_home))
                   end
                   def clear_script(log,problem_home)
                     log.each do |s|
                       FileUtils.rm("#{problem_home}/script/#{s}")
                     end
                   end
                   def mkdir_if_does_not_exist(dirname)
                     Dir.mkdir(dirname) if !FileTest.exist?(dirname)
                   end
                 end
               end

lib/runner.rb ¶ +3

               #
               # A runner drives the engine into various tasks.
               #
               module Grader
                 class Runner
                   def initialize(engine, grader_process=nil)
                     @engine = engine
                     @grader_process = grader_process
                   end
                   def grade_oldest_task
                     task = Task.get_inqueue_and_change_status(Task::STATUS_GRADING)
                     if task!=nil
                       @grader_process.report_active(task) if @grader_process!=nil
                       submission = Submission.find(task.submission_id)
                       @engine.grade(submission)
                       task.status_complete!
                       @grader_process.report_inactive(task) if @grader_process!=nil
                     end
                     return task
                   end
+            +     # grade a specified problem for the latest submission of each user
+            +     # optionally, on all submission when options[:all_sub] is set
+            +     # optionally, only submission that has error (use when the problem itself has some problem)
                   def grade_problem(problem, options={})
                     user_index = 0
                     user_count = User.count
                     User.find_each do |u|
                       puts "user: #{u.login} (#{user_index}/#{user_count})"
                       user_index += 1
                       if options[:user_conditions]!=nil
                         con_proc = options[:user_conditions]
                         next if not con_proc.call(u)
                       end
                       if options[:all_sub]
                         Submission.where(user_id: u.id,problem_id: problem.id).find_each do |sub|
                           next if options[:only_err] and sub.grader_comment != 'error during grading'
                           @engine.grade(sub)
                         end
                       else
                         last_sub = Submission.find_last_by_user_and_problem(u.id,problem.id)
                         if last_sub!=nil
                           @engine.grade(last_sub) unless options[:only_err] and last_sub.grader_comment != 'error during grading'
                         end
                       end
                     end
                   end
                   def grade_submission(submission)
                     puts "Submission: #{submission.id} by #{submission.try(:user).try(:full_name)}"
                     @engine.grade(submission)
                   end
                   def grade_oldest_test_request
                     test_request = TestRequest.get_inqueue_and_change_status(Task::STATUS_GRADING)
                     if test_request!=nil
                       @grader_process.report_active(test_request) if @grader_process!=nil
                       @engine.grade(test_request)
                       test_request.status_complete!
                       @grader_process.report_inactive(test_request) if @grader_process!=nil
                     end
                     return test_request
                   end
                 end
               end

lib/submission_helper.rb ¶ +12 -12

               module Grader
                 class SubmissionRoomMaker
                   def initialize
                     @config = Grader::Configuration.get_instance
                   end
+            +
                   def produce_grading_room(submission)
                     user = submission.user
                     problem = submission.problem
                     grading_room = "#{@config.user_result_dir}/" +
                       "#{user.login}/#{problem.name}/#{submission.id}"
+            +
                     FileUtils.mkdir_p(grading_room)
                     grading_room
                   end
+            +
                   def find_problem_home(submission)
                     problem = submission.problem
                     "#{@config.problems_dir}/#{problem.name}"
                   end
                   def save_source(submission,source_name)
                     dir = self.produce_grading_room(submission)
                     f = File.open("#{dir}/#{source_name}","w")
                     f.write(submission.source)
                     f.close
                   end
                   def clean_up(submission)
                   end
                 end
+            +
                 class SubmissionReporter
                   def initialize(options={})
                     options = {:dry_run => false, :result_collector => nil}.merge(options)
                     @config = Grader::Configuration.get_instance
                     @dry_run = options[:dry_run]
                     @result_collector = options[:result_collector]
                   end
+            +
                   def report(sub,test_result_dir)
                     result = read_result(test_result_dir)
                     if @result_collector
                       @result_collector.save(sub,
                                              result)
                     end
                     save_result(sub,result)
                   end
+            +
                   def report_error(sub,msg)
                     save_result(sub,{:points => 0,
                                   :comment => "Grading error: #{msg}" })
                   end
                   protected
                   def read_result(test_result_dir)
                     cmp_msg_fname = "#{test_result_dir}/compiler_message"
                     if FileTest.exist?(cmp_msg_fname)
                       cmp_file = File.open(cmp_msg_fname)
                       cmp_msg = cmp_file.read
                       cmp_file.close
                     else
                       cmp_msg = ""
                     end
+            +
                     result_fname = "#{test_result_dir}/result"
                     comment_fname = "#{test_result_dir}/comment"
                     runstat_fname = "#{test_result_dir}/run_stat"
                     if FileTest.exist?(result_fname)
                       comment = ""
                       begin
                         result_file = File.open(result_fname)
                         result = result_file.readline.to_i
                         result_file.close
                       rescue
                         result = 0
                         comment = "error reading result file."
                       end
+            +
                       begin
                         comment_file = File.open(comment_fname)
                         comment += comment_file.readline.chomp
                         comment_file.close
                       rescue
                         comment += ""
                       end
                       begin
                         runstat_file = File.open(runstat_fname)
                         max_runtime = runstat_file.readline.to_f
                         peak_memory = runstat_file.readline.to_i
                       rescue
                         max_runtime = -1
                         peak_memory = -1
                       end
                       return {points: result,
                         comment: comment,
                         cmp_msg: cmp_msg,
                         max_runtime: max_runtime,
                         peak_memory: peak_memory
                       }
                     else
                       if FileTest.exist?("#{test_result_dir}/a.out")
                         return {:points => 0,
                           :comment => 'error during grading',
                           :cmp_msg => cmp_msg}
                       else
                         return {:points => 0,
                           :comment => 'compilation error',
                           :cmp_msg => cmp_msg}
                       end
                     end
                   end
+            +
                   def save_result(submission,result)
                     problem = submission.problem
                     submission.graded_at = Time.now.gmtime
                     points = result[:points]
                     submission.points = points
                     comment = @config.report_comment(result[:comment])
                     submission.peak_memory = result[:peak_memory]
                     submission.max_runtime =  result[:max_runtime]
                     submission.effective_code_length =submission.source.length
                     #
                     # TODO: FIX THIS MESSAGE
                     #
                     if problem == nil
                       submission.grader_comment = 'PASSED: ' + comment + '(problem is nil)'
                     elsif points == problem.full_score
                       #submission.grader_comment = 'PASSED: ' + comment
                       submission.grader_comment = comment
                     elsif result[:comment].chomp =~ /^[\[\]P]+$/
                       submission.grader_comment = 'PASSED: ' + comment + '(inconsistent score)'
                     else
                       #submission.grader_comment = 'FAILED: ' + comment
                       submission.grader_comment = comment
                     end
+            +
                     #very lazy trim the string
                     submission.compiler_message = result[:cmp_msg][0..60000] or ''
                     if not @dry_run
                       submission.save
                     end
                   end
+            +
                 end
+            +
               end

Write
Preview

You need to be logged in to leave comments. Login now

	Site-wide shortcuts
/	Use quick search box
g h	Goto home page
g g	Goto my private gists page
g G	Goto my public gists page
n r	New repository page
n g	New gist page

	Repositories
g s	Goto summary page
g c	Goto changelog page
g f	Goto files page
g F	Goto files page with file search activated
g p	Goto pull requests page
g o	Goto repository settings
g O	Goto repository permissions settings

Sign in to your account

Author

r254:8d92eda43960 - Wed, 11 Jul 2018 10:33:10 - 4 files changed: 48 inserted, 41 deleted

Sign in to your account

Commit r254:8d92eda43960 public

Author

r254:8d92eda43960 - Wed, 11 Jul 2018 10:33:10 - 4 files changed: 48 inserted, 41 deleted

Commit `r254:8d92eda43960` public