Skip to content
GitLab
Projects
Groups
Snippets
Help
Loading...
Help
Help
Support
Community forum
Keyboard shortcuts
?
Submit feedback
Contribute to GitLab
Sign in
Toggle navigation
Open sidebar
Pablo Daniel Herrero
metodos-2024-2c-tp2
Commits
8662363c
Commit
8662363c
authored
6 months ago
by
Pablo Herrero
Browse files
Options
Download
Email Patches
Plain Diff
Now that's interesting...
parent
c25b1025
Changes
4
Hide whitespace changes
Inline
Side-by-side
Showing
4 changed files
with
21 additions
and
9 deletions
+21
-9
README.md
README.md
+8
-0
src/classifier/misc.py
src/classifier/misc.py
+9
-5
src/experiments/3d.py
src/experiments/3d.py
+2
-3
src/experiments/full_dataset_analisys.py
src/experiments/full_dataset_analisys.py
+2
-1
No files found.
README.md
View file @
8662363c
...
...
@@ -44,3 +44,11 @@ Ejemplo para grafico de difusion:
$
.
venv/bin/activate
$
python src/experiments/difussion_2d.py 100
```
Para algunos experimentos muy costosos, es posible ejecutarlos en paralelo si el hardware lo permite.
En estos casos, se ejecutaran por defecto, usando el máximo número de CPUs físicos disponibles (sin contar SMT).
Si se quiere asignar un número de cpus distinto a utilizar, hay que setear la variable de entorno
`MAX_CPUS`
antes de ejecutar el script:
```
shell
$ MAX_CPUS
=
2 python src/experiments/3d.py
```
This diff is collapsed.
Click to expand it.
src/classifier/misc.py
View file @
8662363c
import
os
import
subprocess
as
sp
def
get_physical_cores
():
list
=
set
(
map
(
int
,
sp
.
run
([
'lscpu'
,
'-b'
,
'-e=Core'
],
capture_output
=
True
,
text
=
True
)
.
stdout
.
splitlines
()[
1
:]))
try
:
res
=
sp
.
run
([
'/usr/bin/lscpu'
,
'-b'
,
'-e=Core'
],
capture_output
=
True
,
text
=
True
,
check
=
True
)
return
len
(
set
(
map
(
int
,
res
.
stdout
.
splitlines
()[
1
:])))
except
(
FileNotFoundError
,
sp
.
CalledProcessError
):
return
os
.
cpu_count
()
//
2
# type: ignore
return
len
(
list
)
\ No newline at end of file
def
config_for_multiple_cpus
():
return
os
.
getenv
(
'MAX_CPUS'
,
get_physical_cores
())
This diff is collapsed.
Click to expand it.
src/experiments/3d.py
View file @
8662363c
import
multiprocessing
as
mp
from
multiprocessing.pool
import
AsyncResult
import
os
import
pickle
import
numpy
as
np
import
matplotlib.pyplot
as
plt
...
...
@@ -8,10 +7,10 @@ from classifier.data_models import DataModelMatrices, get_vectorized_data_for, g
from
classifier.cv
import
cross_validations
from
classifier.knn
import
KNNPredictor
from
classifier.data_loader
import
*
from
classifier.misc
import
get_physical_core
s
from
classifier.misc
import
config_for_multiple_cpu
s
from
classifier.pca
import
PCA
MAX_PROCESSES
=
int
(
os
.
getenv
(
'MAX_CPUS'
,
get_physical_core
s
()
))
MAX_PROCESSES
=
config_for_multiple_cpu
s
()
MAXIMUM_K
=
100
# Neighbors
P_MIN
=
50
P_MAX
=
250
...
...
This diff is collapsed.
Click to expand it.
src/experiments/full_dataset_analisys.py
View file @
8662363c
...
...
@@ -7,9 +7,10 @@ from classifier.data_models import DataModelMatrices, get_vectorized_data_for, g
from
classifier.cv
import
cross_validations
from
classifier.knn
import
KNNPredictor
from
classifier.data_loader
import
*
from
classifier.misc
import
config_for_multiple_cpus
from
classifier.pca
import
PCA
MAX_PROCESSES
=
8
MAX_PROCESSES
=
config_for_multiple_cpus
()
MAXIMUM_K
=
100
# Neighbors
P_MIN
=
50
P_MAX
=
250
...
...
This diff is collapsed.
Click to expand it.
Write
Preview
Markdown
is supported
0%
Try again
or
attach a new file
.
Attach a file
Cancel
You are about to add
0
people
to the discussion. Proceed with caution.
Finish editing this message first!
Cancel
Please
register
or
sign in
to comment