Windows updates: seconds attempt #248

yeputons · 2014-02-14T16:17:16Z

@tmeasday @seanmwalker @sdarnell I think that it's time to create a new pull request so that we can see all changes in one place. I've disabled meteor pinning for Windows and added some line endings heuristics which tries to preserve type of EOLs in smart.lock, smart.json and .gitignore (os.EOL is used by default). meteorite create-project creates stub project with system-specific EOLs.

No spaces folder:

All tests are passed on Linux (126)
All tests are passed on Windows (123, yellow warning is displayed before tests)

Folder with spaces (it implies that HOME directory have spaces during testing too):

Three meteor-pinning tests are failing on Linux because of old Meteor testing versions (these versions themselves are not able to correctly run from folder with spaces).
Lots of tests are failing on Windows because of If both executable and at least one argument have spaces, nothing works featurist/spawn-cmd#3 (spaces issue). However, creating and working with projects which have spaces in path is OK and works fine, because we run git by its name during regular work, not by its absolute path and therefore this path is not getting quoted.

Things left to do:

update readme with new information about Meteorite on Windows (don't forget to mention that we use directory junctions and some ancient software may handle it strange)
decide what to do with If both executable and at least one argument have spaces, nothing works featurist/spawn-cmd#3 It's actually affects testing on Windows only and has nothing to do with regular usage.

Waiting for your comments, code review and thoughts on the second TODO thing.

…EPATH, and added in the wich package for fining git, curl and meteor. This seems to work reasonably well in cmd.exe and in cygwin. Still working on test issues though.

…s now it's changed iff there was no such env variable before

…using taskkill.exe)

…doesn't do that itself

…s via a shared function for building the full path based on the subdirectory path provided. Updated the tests to call meteorite instead of mrt, so that windows will be able to execute them as well. There are a lot of tests to address to get the windows execution working at 100%.

…ve() to be consistent with what package.js do on Windows (driver letter issue)

Conflicts: spec/acceptance/test_spec.js

… windows consistent in both cygwin and cmd. Changed the mrt.js call to use the meteorite.cmd file created by 'npm install'. This fixed 38 tests for me. - TODO: Refactor how we get the paths for tests, fix the 3 remaining tests and clean up a few odds and ends.

Conflicts: spec/lib/atmosphere.js spec/support/bin/curl

…ome reason)

…ectory is pre-created as a workaround for windows git

…s - it's one of ./bin/mrt.{js,bat} (the latter was added)

…e target path before calling REAL_GIT

…om directories with spaces

…oriteExecutable to work OK from folders with spaces

…ich waits till stdout/stderr buffers are flushed on Windows before exit

…ncRecursive

….js was correspondingly modified

…R initialization

…ch added '.bat' to 'meteor' command before spawn-cmd

…inning

…t.json, smart.lock and .gitignore - original EOL is used or os.EOL of there were no such file

…les instead of '\n'

…tructor and reads both smart.lock and smart.json

Conflicts: lib/meteor.js

…te on Windows and the new 'meteorite' command

Spiralis · 2014-05-20T20:52:31Z

As for all I can say this looks good. Does seem like the second issue might be resolved (featurist/spawn-cmd#3), but the issue is still open it seems.

Hades32 · 2014-05-23T21:41:04Z

The usage of junctions makes it mandatory to run mrt as administrator. This is not very nice and should at least be mentioned in the readme.

"Ordinary" symbolic links can AFAIK be created by normal users, but that has also drawbacks. I think there are some libraries that fake unix style symbolic links, maybe that would be a better approach?

yeputons · 2014-05-29T13:35:39Z

@Hades32 are you sure about directory junctions? It's not the same as symbolic/hard link on Windows. I've just tried to create one under a restricted user and it succeeded, while creation of hard link to file/directory failed because of lack of rights.

Moreover, this branch of Meteorite works just fine under restricted user too.

seanmwalker · 2014-05-29T14:16:40Z

@Hades32 were you using a directory path outside of your users directory?
(A folder outside of C:\users{your username} for instance. You can find your user directory with the USERPROFILE environment variable if you're not sure)

Hades32 · 2014-05-29T17:10:37Z

Well, I tested it that way:

C:\Temp\syn>mklink /D SymbolicDirectory RealDirectory
Ihre Berechtigungen reichen nicht aus, um diesen Vorgang auszuführen. (NO PERMISSION)

C:\Temp\syn>mklink /J JunctionDirectory RealDirectory
Verbindung erstellt für JunctionDirectory <<===>> RealDirectory (SUCCESS)

C:\Temp\syn>mklink /H HardDirectory RealDirectory
Zugriff verweigert (ACCESS DENIED)

So, yes, I'm testing outside of my user directory, but I'm the owner of the directory and I'm administrator (just running an ordinary (not elevated) prompt.

yeputons · 2014-05-29T17:13:03Z

@Hades32 so what's the problem? Meteorite use the second way (directory junctions) and your log says it's OK, isn't it?

Hades32 · 2014-05-30T18:32:06Z

Well, maybe I made some false assumptions there, but my problem essentially is this:

C:\Temp\fahrtenbuch>meteorite

Stand back while Meteorite does its thing
V router
    tag: https://github.com/tmeasday/meteor-router.git#v0.6.1

fs.js:730
  return binding.symlink(preprocessSymlinkDestination(destination, type),
                 ^
Error: EPERM, operation not permitted 'C:\Users\Martin\AppData\Local\.meteorite\packages\router\tmeasday\meteor-router\8ec75fec7affdefc787d19f23b36fd6d1d44ef04'

    at Object.fs.symlinkSync (fs.js:730:18)
    at C:\OtherProgs\meteorite\lib\dependencies\package.js:131:10
    at C:\OtherProgs\meteorite\lib\sources\git.js:84:15
    at GitSource._load (C:\OtherProgs\meteorite\lib\sources\git.js:142:12)
    at C:\OtherProgs\meteorite\lib\sources\git.js:82:18
    at C:\OtherProgs\meteorite\lib\sources\git.js:98:5
    at C:\OtherProgs\meteorite\lib\sources\git.js:264:5
    at ChildProcess.exithandler (child_process.js:645:7)
    at ChildProcess.EventEmitter.emit (events.js:117:20)
    at maybeClose (child_process.js:753:16)

yeputons · 2014-06-02T20:45:11Z

@Hades32 are you able to reproduce this issue in another directory? Say, clean check out of some project in your home directory? Are you able to manually create directory junction between the path Meteorite displayed and corresponding directory in packages/?

Hades32 · 2014-06-09T13:26:45Z

Interestingly I cannot reproduce inside my user directory. As both are owned by the same user I don't really understand why though. I AM able to create a junction between above path and the packages dir - so that doesn't really give a hint either...
But at least I can now comfortably work when I use my user directory - thanks for that!

rudolfb · 2014-07-22T15:22:21Z

@yeputons
Just wanted to thank you for your work in getting mrt working under Windows.

Managed to get your branch working by doing the following on my Windows 8 workstation:

Cloned the oortcloud/meteorite repo
Switched to your branch: https://github.com/oortcloud/meteorite/tree/windows-updates

Needed to install the prerequisite npm packages, and then meteorite:

$ npm install -g ddp,underscore,wrench,fstream,optimist,prompt,colors,async,rolling_timeout_exec,which,exit,spawn-cmd,mocha,connect,meteorite
$ npm install -g meteorite

Had to change some of the package version numbers in the package.json:

, "underscore": "1.3.3"
, "underscore": "1.6.0"
, "prompt": "0.2.11"
, "colors": "0.6.0-1"
, "async": "0.2.9"
, "prompt": "0.2.13"
, "colors": "0.6.2"
, "async": "0.9.0"

After this, I got your code working, as long as theC:\Users\[User]\AppData\Roaming\npm is on one's path.

meteorite did not work for me, but if I use mrt.cmd, e.g. mrt.cmd add luma-ui or simply mrt.cmd, then the commands all seem to work.

Sometimes after issuing a mrt.cmd update it seemed as if the process got hung up. Pressed CTRL-C, then reissued the command and everything was fine. But I had this happen using a meteor update as well, so I doubt this has something to do with your modifications.

All in all it seems to work. Tried these steps on a second PC successfully as well. As long as the C:\Users\rb\AppData\Roaming\npm is on my path, then calling mrt.cmd works fine for me.

Again, thanks.

Regards

Rudolf

Windows PowerShell
Copyright (C) 2013 Microsoft Corporation. Alle Rechte vorbehalten.

PS C:\Users\rb> npm install -g ddp,underscore,wrench,fstream,optimist,prompt,colors,async,rolling_timeout_exec,which,exi
t,spawn-cmd,mocha,connect
C:\Users\rb\AppData\Roaming\npm\which -> C:\Users\rb\AppData\Roaming\npm\node_modules\which\bin\which
C:\Users\rb\AppData\Roaming\npm\_mocha -> C:\Users\rb\AppData\Roaming\npm\node_modules\mocha\bin\_mocha
C:\Users\rb\AppData\Roaming\npm\mocha -> C:\Users\rb\AppData\Roaming\npm\node_modules\mocha\bin\mocha
[email protected] C:\Users\rb\AppData\Roaming\npm\node_modules\rolling_timeout_exec

[email protected] C:\Users\rb\AppData\Roaming\npm\node_modules\which

[email protected] C:\Users\rb\AppData\Roaming\npm\node_modules\spawn-cmd

[email protected] C:\Users\rb\AppData\Roaming\npm\node_modules\colors

[email protected] C:\Users\rb\AppData\Roaming\npm\node_modules\underscore

[email protected] C:\Users\rb\AppData\Roaming\npm\node_modules\async

[email protected] C:\Users\rb\AppData\Roaming\npm\node_modules\wrench

[email protected] C:\Users\rb\AppData\Roaming\npm\node_modules\exit

[email protected] C:\Users\rb\AppData\Roaming\npm\node_modules\fstream
+-- [email protected]
+-- [email protected]
+-- [email protected]
+-- [email protected]

[email protected] C:\Users\rb\AppData\Roaming\npm\node_modules\optimist
+-- [email protected]
+-- [email protected]

[email protected] C:\Users\rb\AppData\Roaming\npm\node_modules\ddp
+-- [email protected]
+-- [email protected]
+-- [email protected] ([email protected])

[email protected] C:\Users\rb\AppData\Roaming\npm\node_modules\connect
+-- [email protected]
+-- [email protected]
+-- [email protected] ([email protected])
+-- [email protected] ([email protected], [email protected])

[email protected] C:\Users\rb\AppData\Roaming\npm\node_modules\mocha
+-- [email protected]
+-- [email protected]
+-- [email protected]
+-- [email protected]
+-- [email protected] ([email protected])
+-- [email protected] ([email protected], [email protected])
+-- [email protected] ([email protected], [email protected], [email protected])

[email protected] C:\Users\rb\AppData\Roaming\npm\node_modules\prompt
+-- [email protected]
+-- [email protected]
+-- [email protected] ([email protected])
+-- [email protected] ([email protected], [email protected], [email protected], [email protected], [email protected], [email protected])
+-- [email protected] ([email protected], [email protected], [email protected], [email protected], [email protected], [email protected])
PS C:\Users\rb> npm install -g meteorite
C:\Users\rb\AppData\Roaming\npm\mrt -> C:\Users\rb\AppData\Roaming\npm\node_modules\meteorite\bin\mrt.js



> [email protected] postinstall C:\Users\rb\AppData\Roaming\npm\node_modules\meteorite
> sh ./completions/postinstall.sh

[email protected] C:\Users\rb\AppData\Roaming\npm\node_modules\meteorite
+-- [email protected]
+-- [email protected]
+-- [email protected]
+-- [email protected]
+-- [email protected]
+-- [email protected] ([email protected], [email protected])
+-- [email protected] ([email protected], [email protected], [email protected], [email protected])
+-- [email protected] ([email protected], [email protected], [email protected])
+-- [email protected] ([email protected], [email protected], [email protected], [email protected], [email protected])
PS C:\Users\rb>

seanmwalker and others added 30 commits January 20, 2014 01:47

Updated paths, added meteorite alias, addressed the HOME env with HOM…

bd7c019

…EPATH, and added in the wich package for fining git, curl and meteor. This seems to work reasonably well in cmd.exe and in cygwin. Still working on test issues though.

#224: spec/acceptance/test_spec.js: ':' --> path.delimiter

5c7e5b5

#224: test_spec.js: METEOR_WAREHOUSE_DIR issue on win32 was fixed plu…

8fde22d

…s now it's changed iff there was no such env variable before

#224: mrt install was fixed: symlinkSync('junction') is used

5f3ce28

#224: spec/lib/runner.js: killProcessFamily() now works on Windows (…

2463d5f

…using taskkill.exe)

#224: now folders are pre-created before 'git clone', git on Windows …

b7367f2

…doesn't do that itself

Moved the seting of path to the bottom.

127805a

#224: smart_lock_spec.js test were fixed (forward/back slashes mess)

91a2dd7

#224: resolver_spec.js: absolute paths were passes through path.resol…

a48f004

…ve() to be consistent with what package.js do on Windows (driver letter issue)

Merge branch 'master' into windows-updates

45623a6

Conflicts: spec/acceptance/test_spec.js

Fixed the 'which' version.

b953136

Merge branch 'master' into windows-updates

decf301

Conflicts: spec/lib/atmosphere.js spec/support/bin/curl

CONTRIBUTING.md: merged with master (looks like it haven't been for s…

a81efd3

…ome reason)

#224: acceptance/test_spec.js now overrides both LOCALAPPDATA and HOME

d9eea8f

spec/support/bin/git: '-T' key was added to 'cp'', because target dir…

c9e0049

…ectory is pre-created as a workaround for windows git

#224: tests: now meteoriteExecutable is calculated in /spec/lib/env.j…

92695e0

…s - it's one of ./bin/mrt.{js,bat} (the latter was added)

#224: spec/support/bin: git.bat was added; git was fixed to pre-creat…

6f9992e

…e target path before calling REAL_GIT

Merge branch 'master' into windows-updates

8ccf78f

#224: test_spec.js: _METEORITE_REAL_CURL is unused now, removing

66f716e

#224: tests: .bat files: quotes was added to make testing possible fr…

8a7d558

…om directories with spaces

#224: spec/lib/atmosphere.js: baseCommand now adds quotes around mete…

bafcc9d

…oriteExecutable to work OK from folders with spaces

#224: process.exit() was replaced by npm 'exit' package (+return), wh…

729ee2a

…ich waits till stdout/stderr buffers are flushed on Windows before exit

#224: spec/support/bin/git: replaced 'cp -Tr' callby wrench.copyDirSy…

eb6e90f

…ncRecursive

#224: bin/mrt.bat was moved to spec/support/bin/mrt.bat; spec/lib/env…

18a2a84

….js was correspondingly modified

windows-updates: replaced require('child_process').spawn by spawn-cmd

7841a32

test_spec.js: removed path.basename() call from _METEORITE_REAL_METEO…

3cd862b

…R initialization

windows-updated: removed temporary Meteor.prototype._executable() whi…

e3ad246

…ch added '.bat' to 'meteor' command before spawn-cmd

windows-updates: lib/meteor.js some outdated comments were removed

84b77c9

yeputons added 8 commits February 14, 2014 18:35

windows-updates: lib/meteor.js: forced defaultMeteor on Windows

5f29f6f

windows-updates: README.txt was added with information about Meteor p…

099aab2

…inning

windows-updates: disabled Meteor pinning tests on Windows

7f4caca

windows-updates: added detectEol() and its calls when writing to smar…

906f56a

…t.json, smart.lock and .gitignore - original EOL is used or os.EOL of there were no such file

windows-updates: now 'create-package' command uses os.EOL for stub fi…

dd7068e

…les instead of '\n'

windows-updates: EOL heuristics: now it's run once, in Project's cons…

0ead4d3

…tructor and reads both smart.lock and smart.json

Merge branch 'master' into windows-updates

a2ec8de

Conflicts: lib/meteor.js

windows-updates: README.md was updated with information about Meteori…

a46be82

…te on Windows and the new 'meteorite' command

yeputons mentioned this pull request May 23, 2014

fs.unlink() treats permissions differently on Windows vs. Unix nodejs/node-v0.x-archive#3006

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Windows updates: seconds attempt #248

Windows updates: seconds attempt #248

yeputons commented Feb 14, 2014

Spiralis commented May 20, 2014

Hades32 commented May 23, 2014

yeputons commented May 29, 2014

seanmwalker commented May 29, 2014

Hades32 commented May 29, 2014

yeputons commented May 29, 2014

Hades32 commented May 30, 2014

yeputons commented Jun 2, 2014

Hades32 commented Jun 9, 2014

rudolfb commented Jul 22, 2014

Windows updates: seconds attempt #248

Are you sure you want to change the base?

Windows updates: seconds attempt #248

Conversation

yeputons commented Feb 14, 2014

Spiralis commented May 20, 2014

Hades32 commented May 23, 2014

yeputons commented May 29, 2014

seanmwalker commented May 29, 2014

Hades32 commented May 29, 2014

yeputons commented May 29, 2014

Hades32 commented May 30, 2014

yeputons commented Jun 2, 2014

Hades32 commented Jun 9, 2014

rudolfb commented Jul 22, 2014